App services manifest API

App services manifest API#

Before you start, make sure that the igz_mgmt package is installed and that you are logged in to the system with the igz_mgmt.Client API. If not, see Control plane API.

import igz_mgmt

client = igz_mgmt.Client(access_key="some-access-id")

Get the app services manifest#

app_services_manifest = igz_mgmt.AppServicesManifest.get(client)

for app_service in app_services_manifest.app_services:
    print(f"Service name: {app_service.spec.name}, kind: {app_service.spec.kind}")

Many of the AppServicesManifest functions create jobs. See more details in Jobs API.

Creating an app service#

Example of creating a Jupyter app service. The jupyter arg is the advance spec for this app service.

jupyter_app_service = igz_mgmt.AppServiceSpec(
    name="my-jupyter",
    owner="admin",
    description="my-description",
    jupyter=igz_mgmt.JupyterSpec(),
)

# create a new app service
app_services_manifest.create_or_update(client, jupyter_app_service)

Resolve the app service#

Example of finding a specific app service from the app services list by app service name. The response includes the service’s spec, status and metadata:

jupyter_service = app_services_manifest.resolve_service("my-jupyter")

# service description
print(f"Jupyter description: {jupyter_service.spec.description}")

# jupyter image name
print(f"Jupyter image name: {jupyter_service.spec.jupyter.image_name}")

# jupyter service state
print(f"Jupyter service state: {jupyter_service.status.state}")

Update the app service#

Update the app service by changing the service spec and calling create_or_update with the new spec. Below is an example of updating the app service description.

jupyter_service.spec.description = "new-description"
app_services_manifest.create_or_update(client, jupyter_service.spec)

# check if the service updated
app_services_manifest = igz_mgmt.AppServicesManifest.get(client)
jupyter_service = app_services_manifest.resolve_service("my-jupyter")
print(f"jupyter description: {jupyter_service.spec.description}")

Restart the app service#

Restart the app service by name. This will restart the app service resources, e.g.: pods.

app_services_manifest.restart(client, jupyter_service.spec.name)

Set apply mode#

Set the apply services mode by igz_mgmt.ApplyServicesMode enum:

app_services_manifest.set_apply_mode(igz_mgmt.ApplyServicesMode.default)

Deleting an app service#

app_services_manifest.remove_service(client, jupyter_service.spec.name)

Apply services#

Run apply_services together with the with statement to perform several batch operations against the app services API.

Create or update services#

The following scenario creates two app services, Jupyter and Shell (or update them if they exist). The Jupyter service uses the defined spec class and env variables. The Shell service uses a custom spec that represents all app services that are yet to be defined by the SDK, and an ssh server.

jupyter_app_service = igz_mgmt.AppServiceSpec(
    name="jupyter-sdk-test",
    owner="admin",
    jupyter=igz_mgmt.JupyterSpec(
        extra_environment_vars={
            "entries": [
                {"key": "dummy-key", "value": "dummy-value"},
            ]
        }
    ),
)

shell_app_service = igz_mgmt.AppServiceSpec(
    name="shell-sdk-test",
    owner="admin",
    shell=igz_mgmt.CustomAppServiceSpec(
        ssh_enabled=True, ssh_server={"force_key_regeneration": False, "port": 30001}
    ),
)

with igz_mgmt.AppServicesManifest.apply_services(client) as asm:
    asm.create_or_update(client, jupyter_app_service)
    asm.create_or_update(client, shell_app_service)

Remove services#

This example removes the two services created in the previous section:

with igz_mgmt.AppServicesManifest.apply_services(client) as asm:
    # get services
    jupyter = asm.resolve_service(jupyter_app_service.name)
    shell = asm.resolve_service(shell_app_service.name)

    # remove 2 services at once
    asm.remove_service(client, jupyter.spec.name)
    asm.remove_service(client, shell.spec.name)

Tip:

  • To apply the configuration on all services you can override the force_apply_all_mode arg: igz_mgmt.AppServicesManifest.apply_services(client, force_apply_all_mode=igz_mgmt.ForceApplyAllMode.enabled).

Scale To / From Zero#

Jupyter supports scaling to/from zero based on its usage. To enable scaling to/from zero, see the folowing example. First update the service that enables scaling to zero, then, once scaled to zero, scale it up again.

import time
from igz_mgmt import AppServiceScaleToZeroSpecPresets

# scales to zero after 1 minute of being idle
jupyter_service.spec.scale_to_zero = AppServiceScaleToZeroSpecPresets.one_minute
app_services_manifest.create_or_update(client, jupyter_service.spec)

# give it some time to scale down ...
time.sleep(2 * 60)

# wake up the service
app_services_manifest.scale_from_zero(client, jupyter_service.spec.name)

Tips:

  • You can choose the window size (time to wait before scaling the service to zero) from the presets given by AppServiceScaleToZeroSpecPresets, and pass it to the scale_to_zero parameter of AppServiceSpec, as in the example above.

  • If this method is called from apply_services (not as single operation) you should change the apply mode before calling the cale_from_zero method:

    • igz_mgmt.ApplyServicesMode.scale_from_zero_only for iguzio version < 3.5.3

    • igz_mgmt.ApplyServicesMode.service_owner_edit for iguzio version >= 3.5.3

    • Obtaining the iguazio version through client.version

Enable / disable app service#

Enable or disable an app service by changing the desired_state field (igz_mgmt.AppServiceDesiredStates enum).

Disable an app service#

jupyter_service.spec.desired_state = igz_mgmt.AppServiceDesiredStates.disabled

Enable an app service#

jupyter_service.spec.desired_state = igz_mgmt.AppServiceDesiredStates.ready

Set the app service priority class#

Change the priority class to low with the igz_mgmt.AppServicePriorityClass enum.

jupyter_service.spec.advanced.priority_class_name = (
    igz_mgmt.AppServicePriorityClass.workload_low
)

Add an app service node selector#

jupyter_service.spec.advanced.node_selector = {
    "entries": [{"key": "dummy-key", "value": "dummy-value"}]
}

Change the app service resources request / limits#

This is an example of changing the Jupyter app service resources request / limits. If not provided, the default values are provided by the platform.

Resource units are required. For example, 100m is 100 millicores, and 1Gi is 1 gigabyte. These units are the same as the Kubernetes units, See more details in the Kubernetes documentation.

Before changing the resources, make sure you have enough resources in your cluster or the app service will not be able to start.

jupyter_service.spec.resources.limits.cpu = "100m"
jupyter_service.spec.resources.limits.memory = "20Gi"

jupyter_service.spec.resources.requests.cpu = "1m"
jupyter_service.spec.resources.requests.memory = "1.2Gi"
jupyter_service.spec.resources.requests.nvidia_gpu = "1"
Change the app service Persistent Volume Claims (PVCs)
jupyter_service.spec.pvc.mounts = {
    "entries": [
        {"my-pvc1": "/my-mount1", "my-pvc2": "/my-mount2"},
    ]
}

Add or modify Jupyter app service environment variables#

jupyter_service.spec.jupyter.extra_environment_vars = {
    "entries": [
        {"key": "dummy-key", "value": "dummy-value"},
    ]
}

Enable an ssh server#

This is an example of enabling an ssh server in the Jupyter app service. The ssh server is disabled by default.

jupyter_service.spec.jupyter.ssh_enabled = True
jupyter_service.spec.jupyter.ssh_server = {
    "force_key_regeneration": False,
    "port": 30001,
}

Update Spark and Trino in the Jupyter app service#

jupyter_service.spec.jupyter.spark_name = "my-spark-service"
jupyter_service.spec.jupyter.presto_name = "my-trino-service"

Set a custom image for the Jupyter app service#

When a custom image is used for jupyter, a docker registry that contains the custom image must also be given.

jupyter_service.spec.jupyter.docker_registry_name = "my-docker-registry-service"
jupyter_service.spec.jupyter.image_name = "my-image-name"

Set the Jupyter image to full stack with GPU#

jupyter_service.spec.jupyter.image_name = (
    igz_mgmt.JupyterAppServicePrebakedImage.gpu_cuda
)

Set the Jupyter image to full stack without GPU#

jupyter_service.spec.jupyter.image_name = (
    igz_mgmt.JupyterAppServicePrebakedImage.full_stack
)

Set the Filebeat Elasticsearch configuration#

filebeat_service = app_services_manifest.resolve_service("log-forwarder")
filebeat_service.spec.filebeat.elasticsearch_url = "dummy-url"
filebeat_service.spec.filebeat.elasticsearch_username = "dummy-username"
filebeat_service.spec.filebeat.elasticsearch_password = "dummy-password"

Set Filebeat Elasticsearch without verification of the server’s certificate#

filebeat_service.spec.elasticsearch_ssl_verification_mode = igz_mgmt.SslVerificationMode.none

Set Filebeat Elasticsearch with full verification of the server’s certificate#

filebeat_service.spec.elasticsearch_ssl_verification_mode = igz_mgmt.SslVerificationMode.full

Set the Filebeat Elasticsearch index template used for the Log Forwarder indices#

filebeat_service.spec.update_index_template = True

Allow manual configuration of the Filebeat Elasticsearch index template#

filebeat_service.spec.update_index_template = False
Contents