Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 6 additions & 3 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,14 @@ when a sandbox create request asks for GPU resources.
|---|---|---|---|
| Docker | Local development with Docker available. | Container plus nested sandbox namespace. | Uses host networking so loopback gateway endpoints work from the supervisor. |
| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API, OCI image volumes, and CDI GPU devices when available. |
| Apple Container | Local macOS development on Apple silicon. | One lightweight Linux VM per sandbox container, plus the nested OpenShell sandbox namespace. | Uses Apple's `container` CLI and a host-reachable supervisor callback endpoint for guest-to-gateway communication. |
| Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | Uses Kubernetes API objects, service accounts, secrets, PVC-backed workspace storage, and GPU resources. |
| VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | Gateway spawns `openshell-driver-vm` as a subprocess over a private, state-local Unix socket. The VM driver boots a cached bootstrap `rootfs.ext4`, prepares requested OCI images inside a bootstrap VM with `umoci`, attaches the prepared image disk read-only, and gives each sandbox a writable `overlay.ext4` for merged-root changes and runtime material. The driver persists each accepted launch request beside the overlay and restarts those VMs on driver startup without recreating the overlay. |

Per-sandbox CPU and memory values currently enter the driver layer through
template resource limits. Docker and Podman apply them as runtime limits.
Kubernetes mirrors each limit into the matching request. VM accepts the fields
but currently ignores them.
template resource limits. Docker, Podman, and Apple Container apply them as
runtime limits. Kubernetes mirrors each limit into the matching request. VM
accepts the fields but currently ignores them.

Docker and Podman also accept per-sandbox driver-config mounts for existing
runtime-managed named volumes and tmpfs mounts. Podman additionally accepts
Expand Down Expand Up @@ -66,6 +67,7 @@ Runtime-specific implementation notes belong in the driver crate README:
- `crates/openshell-driver-docker/README.md`
- `crates/openshell-driver-podman/README.md`
- `crates/openshell-driver-kubernetes/README.md`
- `crates/openshell-driver-apple-container/README.md`
- `crates/openshell-driver-vm/README.md`

## Supervisor Delivery
Expand All @@ -77,6 +79,7 @@ The supervisor must be available inside each sandbox workload:
| Docker | Bind-mounted local supervisor binary, or a binary extracted from the configured supervisor image. |
| Podman | Read-only OCI image volume containing the supervisor binary. |
| Kubernetes | Sandbox pod image or pod template configuration. |
| Apple Container | Host bind-mounted Linux `openshell-sandbox` binary from `supervisor_bin_dir`. |
| VM | Embedded in the guest rootfs bundle. |

Driver-controlled environment variables must override sandbox image or template
Expand Down
5 changes: 4 additions & 1 deletion crates/openshell-bootstrap/src/pki.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ pub struct PkiBundle {
///
/// Covers the host aliases used by every supported runtime: Kubernetes service DNS,
/// `host.docker.internal` for Docker Desktop and rootless Docker on Linux,
/// and `host.containers.internal` for Podman containers reaching their host.
/// `host.containers.internal` for Podman containers reaching their host, and
/// `host.container.internal` for Apple Container guests reaching their host.
pub const DEFAULT_SERVER_SANS: &[&str] = &[
"openshell",
"openshell.openshell.svc",
Expand All @@ -38,6 +39,7 @@ pub const DEFAULT_SERVER_SANS: &[&str] = &[
"*.openshell.localhost",
"host.docker.internal",
"host.containers.internal",
"host.container.internal",
"127.0.0.1",
];

Expand Down Expand Up @@ -190,5 +192,6 @@ mod tests {
fn default_server_sans_include_local_container_hostnames() {
assert!(DEFAULT_SERVER_SANS.contains(&"host.docker.internal"));
assert!(DEFAULT_SERVER_SANS.contains(&"host.containers.internal"));
assert!(DEFAULT_SERVER_SANS.contains(&"host.container.internal"));
}
}
20 changes: 18 additions & 2 deletions crates/openshell-core/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ pub enum ComputeDriverKind {
Vm,
Docker,
Podman,
#[serde(rename = "apple-container")]
AppleContainer,
}

impl ComputeDriverKind {
Expand All @@ -65,6 +67,7 @@ impl ComputeDriverKind {
Self::Vm => "vm",
Self::Docker => "docker",
Self::Podman => "podman",
Self::AppleContainer => "apple-container",
}
}
}
Expand All @@ -84,8 +87,9 @@ impl FromStr for ComputeDriverKind {
"vm" => Ok(Self::Vm),
"docker" => Ok(Self::Docker),
"podman" => Ok(Self::Podman),
"apple-container" => Ok(Self::AppleContainer),
other => Err(format!(
"unsupported compute driver '{other}'. expected one of: kubernetes, vm, docker, podman"
"unsupported compute driver '{other}'. expected one of: kubernetes, vm, docker, podman, apple-container"
)),
}
}
Expand All @@ -94,7 +98,7 @@ impl FromStr for ComputeDriverKind {
/// Auto-detect the appropriate compute driver based on the runtime environment.
///
/// Priority order: Kubernetes → Podman → Docker.
/// VM is never auto-detected (requires explicit `--drivers vm`).
/// VM and Apple Container are never auto-detected and require explicit drivers.
///
/// Returns the first driver where the environment check passes.
/// Returns `None` if no compatible driver is found.
Expand Down Expand Up @@ -795,12 +799,24 @@ mod tests {
"docker".parse::<ComputeDriverKind>().unwrap(),
ComputeDriverKind::Docker
);
assert_eq!(
"apple-container".parse::<ComputeDriverKind>().unwrap(),
ComputeDriverKind::AppleContainer
);
assert_eq!(
" Apple-Container ".parse::<ComputeDriverKind>().unwrap(),
ComputeDriverKind::AppleContainer
);
}

#[test]
fn compute_driver_kind_rejects_unknown_values() {
let err = "firecracker".parse::<ComputeDriverKind>().unwrap_err();
assert!(err.contains("unsupported compute driver 'firecracker'"));
let err = "apple_container".parse::<ComputeDriverKind>().unwrap_err();
assert!(err.contains("unsupported compute driver 'apple_container'"));
let err = "apple".parse::<ComputeDriverKind>().unwrap_err();
assert!(err.contains("unsupported compute driver 'apple'"));
}

#[test]
Expand Down
5 changes: 3 additions & 2 deletions crates/openshell-core/src/driver_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,9 @@ pub const SUPERVISOR_CONTAINER_BINARY: &str = "/opt/openshell/bin/openshell-sand
// ---------------------------------------------------------------------------
// In-container mount paths for guest TLS materials and the sandbox token.
//
// All container-based drivers (Docker, Podman, Kubernetes) mount the gateway's
// mTLS client credentials at these fixed paths inside every sandbox container.
// All container-based drivers (Docker, Podman, Apple Container, Kubernetes)
// mount the gateway's mTLS client credentials at these fixed paths inside every
// sandbox container.
// The supervisor reads these paths on startup to establish its gRPC-over-mTLS
// connection back to the gateway. The paths must remain stable across driver
// versions since the supervisor binary is built and packaged separately.
Expand Down
24 changes: 24 additions & 0 deletions crates/openshell-core/src/telemetry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ pub enum TelemetryComputeDriver {
Kubernetes,
Podman,
Vm,
AppleContainer,
Unknown,
}

Expand All @@ -172,6 +173,7 @@ impl TelemetryComputeDriver {
Self::Kubernetes => "kubernetes",
Self::Podman => "podman",
Self::Vm => "vm",
Self::AppleContainer => "apple-container",
Self::Unknown => "unknown",
}
}
Expand All @@ -183,6 +185,7 @@ impl TelemetryComputeDriver {
"k8s" | "kubernetes" => Self::Kubernetes,
"podman" => Self::Podman,
"vm" => Self::Vm,
"apple-container" => Self::AppleContainer,
_ => Self::Unknown,
}
}
Expand All @@ -194,6 +197,7 @@ impl TelemetryComputeDriver {
Some(crate::ComputeDriverKind::Kubernetes) => Self::Kubernetes,
Some(crate::ComputeDriverKind::Podman) => Self::Podman,
Some(crate::ComputeDriverKind::Vm) => Self::Vm,
Some(crate::ComputeDriverKind::AppleContainer) => Self::AppleContainer,
None => Self::Unknown,
}
}
Expand Down Expand Up @@ -695,6 +699,26 @@ mod tests {
TelemetryComputeDriver::from_raw("podman").as_str(),
"podman"
);
assert_eq!(
TelemetryComputeDriver::from_raw("apple-container").as_str(),
"apple-container"
);
assert_eq!(
TelemetryComputeDriver::from_raw("apple_container").as_str(),
"unknown"
);
assert_eq!(
TelemetryComputeDriver::from_raw("apple").as_str(),
"unknown"
);
assert_eq!(
TelemetryComputeDriver::from_raw("Apple-Container").as_str(),
"apple-container"
);
assert_eq!(
TelemetryComputeDriver::from_raw(" apple-container ").as_str(),
"apple-container"
);
assert_eq!(
TelemetryComputeDriver::from_raw("private-driver").as_str(),
"unknown"
Expand Down
30 changes: 30 additions & 0 deletions crates/openshell-driver-apple-container/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[package]
name = "openshell-driver-apple-container"
description = "Apple Container compute driver for OpenShell"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
repository.workspace = true

[dependencies]
openshell-core = { path = "../openshell-core", default-features = false }

tokio = { workspace = true }
tonic = { workspace = true }
futures = { workspace = true }
tokio-stream = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
tracing = { workspace = true }
chrono = { version = "0.4", default-features = false, features = ["std"] }

[dev-dependencies]
prost-types = { workspace = true }

[lints]
workspace = true
28 changes: 28 additions & 0 deletions crates/openshell-driver-apple-container/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# OpenShell Apple Container Driver

This crate implements the OpenShell compute driver for Apple's `container` CLI.
It creates local macOS sandboxes as Linux containers inside Apple Container
lightweight VMs.

The driver intentionally shells out to the installed `container` CLI instead of
linking Swift or XPC APIs directly. Apple Container's public, supported operator
surface is the CLI, and the CLI exposes machine-readable JSON for the state that
OpenShell needs:

- `container system status --format json`
- `container list --all --format json`
- `container network list --format json`

The gateway must run on macOS with Apple Container installed and running. Set
`compute_drivers = ["apple-container"]` in `[openshell.gateway]`; the gateway
does not auto-detect this driver.

When `grpc_endpoint` is empty, the driver builds the supervisor callback URL
from `host_callback_host` and the gateway bind port. The default callback host
is `host.container.internal`, which Apple Container resolves inside the guest
VM. The gateway also listens on the Apple Container default network gateway
address discovered from `container network list --format json`.

Apple Container accepts integer CPU counts. OpenShell therefore rejects
per-sandbox CPU limits such as `500m` or `1.5` that cannot be passed to
`container run --cpus`.
Loading
Loading