Local Service Operations¶

This page discusses operations relevant to local service management. Please read Local Service States and Operations and Local Service Instances before running lifecycle operations.

Emergency deactivation behavior¶

Drove has an internal emergency deactivation safety path for local services.

• If adjustment/replacement style operations are cancelled while a service is ACTIVE or CONFIG_TESTING, state can move to EMERGENCY_DEACTIVATION_REQUESTED.
• In this state, controller automatically submits a DEACTIVATE operation to move service to a safe inactive state.
• While in EMERGENCY_DEACTIVATION_REQUESTED, only DEACTIVATE is accepted by command validation.

How to initiate an operation¶

All local service lifecycle operations are issued via POST calls to /apis/v1/localservices/operations on the leader controller endpoint.

Sample API call:

curl --location 'http://drove.local:7000/apis/v1/localservices/operations' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--data '{
    "type": "DEACTIVATE",
    "serviceId": "NODE_EXPORTER-1"
}'

Cluster Operation Specification¶

When an operation is submitted to the cluster, a cluster op spec needs to be specified. This is needed to control different aspects of the operation, including parallelism of an operation or increase the timeout for the operation and so on.

The following aspects of an operation can be configured:

How to cancel an operation¶

Operations can be requested to be cancelled asynchronously by calling /apis/v1/localservices/operations/{serviceId}/cancel.

curl --location --request POST 'http://drove.local:7000/apis/v1/localservices/operations/NODE_EXPORTER-1/cancel' \
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--data ''

Create a local service¶

Creates local service metadata and starts instances based on instancesPerHost.

Preconditions:

• Service id ({name}-{version}) must not already exist.
• Spec placement policy must be LOCAL.
• If placementPolicy.hostLevel=true, instancesPerHost must be 1.

State Transition:

• none -> INIT -> ACTIVATION_REQUESTED -> ACTIVE

drove -c local localservices create sample/test_service.json --instances 1

{
    "type": "CREATE",
    "spec": { ... },
    "instancesPerHost": 1
}

Activate a local service¶

Starts instances for a previously inactive local service.

Preconditions:

• Service should be in INACTIVE activation state.

State Transition:

• INACTIVE -> ACTIVATION_REQUESTED -> ACTIVE

drove -c local localservices activate NODE_EXPORTER-1

{
    "type": "ACTIVATE",
    "serviceId": "NODE_EXPORTER-1"
}

Deactivate a local service¶

Stops local service instances without removing service metadata.

Preconditions:

• Service should exist.

State Transition:

• ACTIVE -> DEACTIVATION_REQUESTED -> INACTIVE

drove -c local localservices deactivate NODE_EXPORTER-1

{
    "type": "DEACTIVATE",
    "serviceId": "NODE_EXPORTER-1"
}

Update instances per host¶

Updates desired instance count on each executor.

Preconditions:

• Service must be in ACTIVE state.
• Not allowed for host-level services (placementPolicy.hostLevel=true).

State Transition:

• ACTIVE -> UPDATING_INSTANCES_COUNT -> ADJUSTING_INSTANCES -> ACTIVE

drove -c local localservices update NODE_EXPORTER-1 2

{
    "type": "UPDATE_INSTANCE_COUNT",
    "serviceId": "NODE_EXPORTER-1",
    "instancesPerHost": 2
}

Adjust instances¶

Triggers a reconciliation pass to align actual running instances with desired instancesPerHost.

Preconditions:

• Service must exist.

{
    "type": "ADJUST_INSTANCES",
    "serviceId": "NODE_EXPORTER-1",
    "opSpec": {
        "timeout": "5m",
        "parallelism": 16,
        "failureStrategy": "STOP"
    }
}

Restart a local service¶

Restarts all instances for a service.

For host-level services (hostLevel=true), use stopFirst=true.

Preconditions:

• Service must be in ACTIVE or CONFIG_TESTING state.

drove -c local localservices restart NODE_EXPORTER-1 --parallelism 16 --timeout 5m

{
    "type": "RESTART",
    "serviceId": "NODE_EXPORTER-1",
    "stopFirst": true,
    "opSpec": {
        "timeout": "5m",
        "parallelism": 16,
        "failureStrategy": "STOP"
    }
}

Replace specific instances¶

Replaces selected instances with fresh ones.

For host-level services (hostLevel=true), set stopFirst=true.

Preconditions:

• Service must be in ACTIVE or CONFIG_TESTING state.
• If instanceIds are provided, each id must correspond to a healthy instance.

drove -c local lsinstances replace NODE_EXPORTER-1 SI-123 SI-456 --parallelism 4

{
    "type": "REPLACE_INSTANCES",
    "serviceId": "NODE_EXPORTER-1",
    "instanceIds": ["SI-123", "SI-456"],
    "stopFirst": true,
    "opSpec": {
        "timeout": "5m",
        "parallelism": 4,
        "failureStrategy": "STOP"
    }
}

Stop specific instances¶

Stops selected instances for a local service.

Preconditions:

• Service must be in ACTIVE or CONFIG_TESTING state.
• Each instanceId must correspond to a healthy instance.

drove -c local lsinstances kill NODE_EXPORTER-1 SI-123 --parallelism 1

{
    "type": "STOP_INSTANCES",
    "serviceId": "NODE_EXPORTER-1",
    "instanceIds": ["SI-123"],
    "opSpec": {
        "timeout": "5m",
        "parallelism": 1,
        "failureStrategy": "STOP"
    }
}

Deploy test instance¶

Deploys a test instance for validation workflows.

Preconditions:

• Service must be in INACTIVE state.

drove -c local localservices conftest NODE_EXPORTER-1

{
    "type": "DEPLOY_TEST_INSTANCE",
    "serviceId": "NODE_EXPORTER-1"
}

Destroy a local service¶

Permanently removes local service metadata and remaining records.

Preconditions:

• Service must be in INACTIVE state.

State Transition:

• INACTIVE -> DESTROY_REQUESTED -> DESTROYED -> none

drove -c local localservices destroy NODE_EXPORTER-1

{
    "type": "DESTROY",
    "serviceId": "NODE_EXPORTER-1"
}