# Glassly OEM Integration Engine API

> The engine.* API an OEM host calls to drive Glassly on its glasses — connection, pairing, device + user settings, wifi, speech, display, cloud session, miniapps, permissions, notifications, and reports, over a device-agnostic runtime.

Source: https://docs.dev.glassly.ai/glasses-oems/engine/

**Status: preview.** The Glassly OEM Integration Engine covers the full surface
documented below. The runtime is not yet packaged for external install —
reach out to [help@glassly.ai](mailto:help@glassly.ai) to integrate today.

## What it is

The **Glassly OEM Integration Engine** is the API an OEM host app (the phone-side app
that talks to your glasses) calls to run Glassly. It ships as the `@glassly/engine` module and
exposes a single namespaced object, **`engine`**:

```ts
import {engine} from "@glassly/engine"
```

The dividing line:

- **The host owns UI, navigation, and login.** Your screens, your router, your auth.
- **The engine owns the runtime and the device.** Connection, device state, wifi,
  on-device speech models, the display pipeline, the miniapp runtime.

So an OEM builds its own UI and calls `engine.<domain>.<method>()` for everything
device- and runtime-related. The engine is device-agnostic: the same calls work
across glasses models.

## Lifecycle

The host hands the engine its auth + config once, then starts it.

```ts
import {engine} from "@glassly/engine"

engine.configure({
  // REQUIRED — the host owns login; the engine owns the rest.
  auth: {
    // Return your current (auto-refreshed) backend token.
    getSubjectToken: async () => ({token: await getToken(), type: "supabase"}),
  },
  // Optional cloud endpoints + OEM identity.
  config: {coreUrl, runtimeUrl, oemId},
  // Optional analytics sink.
  analytics: (event, props) => track(event, props),
})

await engine.start() // idempotent
// …
await engine.stop()  // idempotent
```

| Call | Purpose |
|---|---|
| `engine.configure(opts)` | Hand the engine `auth` (required), `config`, `analytics`. Call once, before `start()`. |
| `engine.start()` | Mark the runtime started. Idempotent. |
| `engine.stop()` | Tear down. Idempotent. |

`auth.getSubjectToken()` is the only must-have seam — the host owns login and returns
a fresh token on demand; the engine owns everything downstream.

## `engine.glasses`

Connection actions, a curated status/info read-model, capabilities, version, and the
discrete input events.

### Connection

```ts
await engine.glasses.connectDefault()   // connect the last-paired glasses
await engine.glasses.connect(device)    // connect a specific (discovered) device
await engine.glasses.connectSimulated() // built-in simulated glasses (dev)
await engine.glasses.setDefault(device) // make it the connectDefault() target
await engine.glasses.disconnect()
await engine.glasses.forget()
```

### Controllers

Controllers (R1 rings, Meta Neural Bands) are a registry: several can be paired and
connected at once, two of the same model included. Each is addressed by its
**controller id**, which stays the same while it is paired (the R1's ring id; the
band's peripheral identifier on iOS, its Bluetooth address on Android). Pairing goes
through [`engine.pairing`](#engine-pairing) with a controller model; the native layer
records the controller when it is ready.

```ts
const paired = engine.glasses.controllers.list() // PairedController[] {id, model, name}, pairing order
const offList = engine.glasses.controllers.onList((list) => { … })

const all = engine.glasses.controllers.statuses() // Record<id, ControllerStatus>; absent id = idle
const one = engine.glasses.controllers.status(id)
const offStatus = engine.glasses.controllers.onStatus((statuses) => { … })
const live = engine.glasses.controllers.connected() // connected && fullyBooted
engine.glasses.controllers.isConnected("Meta Neural Band") // without a model: any controller

await engine.glasses.controllers.connect(id)
await engine.glasses.controllers.connectAll() // every paired controller not already connected
await engine.glasses.controllers.cancelAttempt() // drop an in-flight pairing, registry untouched
await engine.glasses.controllers.disconnect(id) // stays paired
await engine.glasses.controllers.forget(id) // unpair
```

A Meta Neural Band also needs the setup in [`engine.glasses.neuralBand`](#engine-glasses-neuralband).

### Status & info (read-models)

`status()` returns a snapshot projected from the runtime's glasses store, in a stable
shape that doesn't leak the internal store layout:

```ts
const s = engine.glasses.status()
// { state, fullyBooted, battery, charging,
//   case: {battery, charging, open, removed},
//   signal, micEnabled, vadEnabled, btClassic }

const unsubscribe = engine.glasses.onStatus((s) => render(s))

const info = engine.glasses.info()
// { model, style, color, firmwareVersion, mtkFirmware, besFirmware,
//   serialNumber, buildNumber, btMac, sceneCanvasWidth, sceneCanvasHeight,
//   volumeGate, firmwareUpdateAvailable }

const caps = engine.glasses.capabilities() // model capability table
await engine.glasses.requestVersionInfo()  // ask glasses to report fresh version info
```

`volumeGate` means the connected firmware can gate microphone streaming (G2 Glassly
CFW revision 40+). `firmwareUpdateAvailable` means a G2 or RayNeo iO runs stock
firmware or an older Glassly CFW revision than the one the SDK bundles; it clears when
the connection resets.

### Input events

```ts
const offButton = engine.glasses.onButtonPress((e) => { /* ButtonPressEvent */ })
const offTouch  = engine.glasses.onTouchGesture((e) => { /* TouchEvent */ })
```

Every `on*()` returns an unsubscribe function.

Head position is a touch gesture: `head_up` and `head_down` arrive through
`onTouchGesture` and reach miniapps under the same gesture claims as taps (the dashboard
miniapp claims both while it runs). An unclaimed `head_up` on glasses whose capabilities
list the `dashboard` view opens the glasses' own dashboard.

### `engine.glasses.neuralBand`

The first time a Meta Neural Band connects to a phone, it runs a one-time claim
against the wearer's Meta account. Your host signs the wearer in to Meta and hands
the session over before connecting the controller:

```ts
await engine.glasses.neuralBand.setMetaSession(accessToken, userId)
await engine.glasses.neuralBand.hasMetaSession()
await engine.glasses.neuralBand.clearMetaSession()
await engine.glasses.neuralBand.isClaimed(id) // a remembered band reconnects without a claim

const claim = engine.glasses.neuralBand.claimStatus() // the latest claim: {stage, message?, sessionExpired, bandName?, controllerId}
const offClaim = engine.glasses.neuralBand.onClaimStatus((s) => { … })
engine.glasses.neuralBand.resetClaim() // back to idle before a new pairing attempt

// Handwriting runs by itself while the neural_band_handwriting setting is on and a
// miniapp has a text field open (input.setTextFieldOpen); this switches it by hand.
await engine.glasses.neuralBand.setHandwriting(id, true) // writing reaches miniapps as onKey presses
const hw = engine.glasses.neuralBand.handwritingStatus(id) // {phase, message, problem?, text}
const offHw = engine.glasses.neuralBand.onHandwritingStatus(({id, status}) => { … })
await engine.glasses.neuralBand.setPointer(id, true) // the air mouse, delivered to input.onPointer
await engine.glasses.neuralBand.setHand(id, "left")
```

`id` is the band's controller id. `stage` is `idle`, `progress`, `completed`, or
`failed`; the claim status is the latest one, since pairing runs before the host knows
the new band's id. `sessionExpired` means Meta refused the saved session: sign the
wearer in again. The per-band calls reject with `no_such_controller` unless `id` is a
connected band. The `neural_band_air_mouse` and `neural_band_hand` settings apply to
every connected band.

### `engine.glasses.wifi`

```ts
const networks = await engine.glasses.wifi.scan()            // WifiSearchResult[]
await engine.glasses.wifi.connect(ssid, password)            // rejects with a coded error on failure
await engine.glasses.wifi.forget(ssid)
const status = engine.glasses.wifi.status()                  // WifiStatus snapshot
const off = engine.glasses.wifi.onStatus((status) => { … })  // subscribe
```

`connect()` propagates the bluetooth layer's coded errors (`bluetooth_powered_off`,
`request_timeout`, …) unchanged, so your UI keeps its own error mapping.

### `engine.glasses.settings`

Keyed **device** settings (brightness, head-up angle, dashboard, camera/button,
sensing, …). `set()` persists *and* auto-syncs to the connected glasses.

```ts
engine.glasses.settings.get(key)
engine.glasses.settings.set(key, value)        // also pushes to the device
engine.glasses.settings.onChanged(key, (v) => { … })
engine.glasses.settings.descriptor(key)
engine.glasses.settings.available()            // the device-setting keys (live list)
```

`available()` is the authoritative list at runtime (it varies by model/capabilities).
The current keys, their types, and defaults:

| Key | Type | Default | Notes / values |
|---|---|---|---|
| `brightness` | number | `50` | display brightness, `0`–`100` |
| `auto_brightness` | boolean | `true` | auto-adjust brightness |
| `head_up_angle` | number | `45` | head-up activation angle in degrees, `0`–`60` |
| `screen_disabled` | boolean | `false` | turn the display off |
| `dashboard_height` | number | `4` | dashboard vertical position |
| `dashboard_depth` | number | `2` | dashboard distance |
| `menu_apps` | `string[] \| null` | `null` | button-menu app package names |
| `button_photo_size` | enum | `"max"` | `low` (960×720) \| `medium` (1440×1088) \| `high` (3264×2448) \| `max` (camera maximum) |
| `button_video_settings` | object | `{ width: 1920, height: 1080, fps: 30 }` | button video recording |
| `button_max_recording_time` | number | `10` | max button-triggered video length, in **minutes** (e.g. `3`, `5`, `10`, `15`, `20`) |
| `camera_fov` | object | `{ fov: 118, roi_position: 0 }` | persistent camera field of view base |
| `preferred_mic` | enum | `"auto"` | `auto` \| `glasses` \| `phone` |
| `lc3_frame_size` | enum (bytes) | `60` | `20` \| `40` \| `60` |
| `sensing_enabled` | boolean | `true` | onboard sensors |
| `voice_activity_detection_enabled` | boolean | `true` | voice-activity detection |
| `loudness_gate_enabled` | boolean | `true` | Mentra Live center-mic loudness ("Barrier") gate |
| `volume_gate_enabled` | boolean | `true` | G2 Glassly CFW (rev 40+, reported as `volumeGate` in the glasses status): the temples stop streaming mic audio while it stays below the threshold. Held open while `loudness_gate_enabled` is off or the Microphone settings level meter is reading raw PCM; miniapps reading raw PCM (such as the recorder) get the gated stream |
| `volume_gate_threshold_db` | number | `-45` | volume gate: dBFS level that counts as sound |
| `volume_gate_hold_ms` | number | `5000` | volume gate: quiet time before it closes |
| `volume_gate_min_open_ms` | number | `10000` | volume gate: minimum time it stays open once sound opens it |
| `power_saving_mode` | boolean | `false` | low-power mode |
| `offline_mode` | boolean | `false` | offline operation |
| `gallery_mode` | boolean | `true` | capture-to-gallery enabled — when `true`, a camera-button press takes a photo to the on-device gallery. The Glassly App auto-manages this (enabled when the camera app is running or no other app is claiming button events); set it directly only if your host has no equivalent button-routing logic |
| `calendar_events` | array | `[]` | synced calendar events |
| `metric_system` | boolean | `false` | metric vs imperial |
| `cfw_scene_renderer` | boolean | `true` | Glassly CFW (G2, RayNeo iO): render scenes through the direct-framebuffer / page path; off falls back to the stock display commands |
| `cfw_fast_ble` | boolean | `false` | G2 Glassly CFW (rev 26+): ask for the 7.5 ms / latency-0 BLE profile instead of the stock 15–30 ms link. Costs battery |
| `g2_perf_logs` | boolean | `false` | debug: log the G2's display performance samples |
| `ring_via_glasses` | boolean | `false` | debug: link the R1 ring to the G2 and take its input through the Glassly CFW relay. Off, the ring stays on its own phone link whether or not a G2 is paired |
| `display_max_fps` | number | `8` | cap on display updates leaving the phone, `2`–`20`. No effect on G2 CFW, whose driver coalesces off the link |
| `nex_chinese_captions` | boolean | `false` | Mentra Display: Chinese captions |
| `nex_lc3_audio_playback` | boolean | `false` | Mentra Display: play back received LC3 audio (Android only) |

Device **identity** (paired-device id/name/address, the paired controller registry) and internal
**runtime flags** (e.g. STT-fallback state) are intentionally *not* in this surface —
read the connected device via `engine.glasses.info()` and the controllers via
`engine.glasses.controllers.list()`.

## `engine.pairing`

First-time glasses discovery + pairing. (Reconnecting the already-paired default is
`engine.glasses.connectDefault()`.)

```ts
await engine.pairing.scan(model)               // start scanning for a DeviceModel
engine.pairing.scanning()                      // is a scan in progress?
engine.pairing.searchResults()                 // discovered devices (snapshot)
const off = engine.pairing.onFound((r) => { … }) // subscribe; returns unsubscribe
await engine.pairing.pair(device)              // connect to a discovered device (from searchResults)
await engine.pairing.setDefault(device)        // make it the connectDefault() target
engine.pairing.onPairFailure((e) => { … })
engine.pairing.onGlassesNotReady((e) => { … })
```

## `engine.speech`

On-device STT/TTS model management.

```ts
engine.speech.stt.currentLanguage()
engine.speech.stt.languages()
engine.speech.stt.languageInfo()
await engine.speech.stt.download(/* model args */)
engine.speech.stt.activate(code)
engine.speech.stt.cancelDownload()
await engine.speech.stt.deleteModel(/* model args */)
engine.speech.stt.status()                       // offline-model auto-download status
engine.speech.stt.onStatusChanged((s) => { … })  // subscribe; returns unsubscribe

// engine.speech.tts mirrors stt (no auto-download status stream).
```

## `engine.display.mirror`

A typed read facade for a phone-side preview of the glasses screen.

```ts
const event = engine.display.mirror.current()             // current display event snapshot
const off   = engine.display.mirror.onMirror((e) => { … }) // subscribe; returns unsubscribe
```

## Scene rendering, shapes, and the canvas

Miniapps draw through the engine's scene API; the engine rasterizes or translates each
frame for the connected device. Two things vary by firmware, and both are read off the
device's effective capabilities rather than hard-coded.

**Vector shapes.** Firmware that draws vectors declares
`rect · circle · arc · pie · line · triangle · quad · bezier`, plus `svg` on Glassly
CFW. Elements carry a style — `border`/`fill`/`radius` on closed shapes, `width` on
open strokes (line, bezier, arc) — and a gray `color`. Devices without vector support
render scenes through the host degrade path instead.

**Intensity.** The baseline is 4-bit grayscale: every drawable's `color` is a gray level
`0`–`15`, where `15` is brightest and `0` means "not drawn". The G2 (stock and CFW),
RayNeo iO and Mentra display all report `intensityLevels: 16`.

**Animation.** An element with a stable `id` can carry a `transition`
(`{durationMs, easing}`, easing one of `linear · ease · ease-in · ease-out · ease-in-out`).
The device tweens the geometry change itself when its capabilities declare `animation`;
on images the tween reads as a fade. `rotation` is a separate capability covering
geometric and SVG rotations.

**Depth.** Devices with two lenses may declare `depth`: per-element `depth` (a
disparity of -16..16 canvas px) is then drawn `depth` px further right on the left
lens and further left on the right, so positive values float in front of the frame.
Glassly CFW declares it and sends each lens its own object-cache SHOW once a frame
uses depth; from G2 revision 41 both SHOWs travel as one stream, so the two lenses start
their animations together. Devices without it draw every element flat; depth never drops anything.

**Canvas negotiation.** The drawable canvas is not fixed. `576×288` is the pre-report
fallback — the stock G2 scene canvas, which firmware blits 1:1 into the 640×480 panel.
Glassly CFW rasterizes scenes phone-side straight into the panel and publishes its real
canvas in the glasses status; the engine then overrides the model profile with the
reported size for any model that bundles a Glassly CFW (the RayNeo iO publishes its
540×180 page the same way). Because CFW frames are not bound by the stock container pools
(6 text / 4 image), element budgets are lifted alongside the canvas: **24 text / 16
image** on a raster CFW build, and **120** shared elements on a CFW build that reports
vector support (shapes and images each cost one cached slot, so shape-heavy frames get
one larger budget).

Read the budgets and canvas from `engine.glasses.capabilities()` at runtime. They change
when the glasses reconnect with different firmware — never bake `576×288` or the stock
pools into a host.

## `engine.session`

The cloud live-session surface. engine owns the cloud client — it
constructs it from engine-owned transports + the `auth` you passed to
`configure()` — so this reads the session it manages.

```ts
const s = engine.session.status()            // { status, audioTransport }
const off = engine.session.onStatus((s) => { … }) // subscribe; returns unsubscribe
const live = engine.session.isConnected()    // handshake completed?
```

`status` is one of `connected | connecting | reconnecting | disconnected`;
`audioTransport` is `udp | ws | offline | none`.

## `engine.settings`

Typed keyed user settings over the engine-owned settings store.

```ts
engine.settings.get(key)               // current value (or default)
engine.settings.set(key, value)        // also syncs to backend (pass false to skip)
engine.settings.onChanged(key, (v) => { … }) // subscribe; returns unsubscribe
engine.settings.descriptor(key)        // schema (type/default/options)
engine.settings.keys()                 // all known keys
```

## `engine.reports`

Bug report and feedback submission. The OEM writes its own report screen,
trigger labels, rating controls, and screenshot picker UX; engine owns runtime
context collection, recent phone logs, Cloud submission, artifact upload,
dedupe, and glasses notification.

```ts
const bug = await engine.reports.submit({
  kind: "bug",
  trigger,
  report,
  screenshots,
})

const automatic = await engine.reports.submit({
  kind: "automatic",
  trigger: {type: "automatic", source, reason},
  report,
  screenshots,
  dedupeKey,
})

const feedback = await engine.reports.submit({
  kind: "feedback",
  feedback: {type, message, experienceRating},
})
```

## `engine.dev`

Developer/debug surface — backend + cloud URL overrides, reconnect, version gate.

```ts
engine.dev.minimumClientVersion()
engine.dev.backendUrl(); engine.dev.setBackendUrl(url)
engine.dev.cloudUrls(); engine.dev.setCloudUrls({core, runtime}) // sets + reconnects
engine.dev.savedUrls()
engine.dev.reconnectCloud()
```

## `engine.permissions`

OS permissions — the raw check/request ops (your UI adds the rationale dialogs).

```ts
await engine.permissions.check("microphone")   // granted?
await engine.permissions.request("location")   // -> granted?
await engine.permissions.openSettings()        // open OS settings
await engine.permissions.requirementsForMiniapp(pkg) // -> AppletPermission[]
```

Feature keys: `microphone · camera · calendar · location · background_location · bluetooth · phone_state · post_notifications`.

## `engine.phoneNotifications`

Forward phone notifications to the glasses, with a per-app blocklist. **Android-only**
(getters return safe defaults on iOS).

```ts
engine.phoneNotifications.enabled(); engine.phoneNotifications.setEnabled(true)
await engine.phoneNotifications.installedApps()      // [{packageName, appName, icon}]
engine.phoneNotifications.blocklist(); engine.phoneNotifications.setBlocklist([...])
await engine.phoneNotifications.hasListenerPermission()
await engine.phoneNotifications.requestListenerPermission()
```

Miniapps post notifications of their own (`session.phone.notifications.post`).
Subscribe with `engine.phoneNotifications.onMiniappNotification((event) => …)`
and feed each `posted` / `dismissed` event to the same path as the phone's, so
readers receive it with `system: false` and `source` set to the miniapp's
package.

## `engine.notifications`

Inbound alerts engine→host — conditions engine detects, your UI renders.

```ts
const off = engine.notifications.onNotification((n) => {
  // n.kind: "miniapp_crashloop" | "version_incompatible" | "connection_failed_persistent"
  // n.packageName, n.reason, n.timestamp, n.metadata
})
```

## `engine.ota`

Firmware OTA. Engine owns availability checks and install orchestration; the host
progress screen is a pure renderer over the coordinator snapshot.

```ts
await engine.ota.initialize()                 // start the status + OTA projections (Bluetooth-only hosts)
await engine.ota.checkForUpdates()            // resolve the manifest, compare, update the snapshot
engine.ota.updateAvailable()                  // OtaUpdateInfo | null
engine.ota.status()                           // OtaStatus | null (stepType/phase/percent/error)
engine.ota.snapshot()                         // OtaSnapshot read-model
await engine.ota.install(manifestUrl?)        // start the firmware install
engine.ota.getReleaseChangelogs(from, to)     // bundled changelogs crossed between two versions
engine.ota.clearUpdateAvailable(); engine.ota.clearProgress()

const off = engine.ota.onUpdateAvailable((info) => { … }) // also onStatus / onSnapshot
```

`engine.ota.installSession` fronts the install state machine for a progress screen:
`prepare(result)`, `attach()`, `detach()`, `retry()`, `finish()`, `discard()`,
`snapshot()`, `onSnapshot(cb)`. Every watchdog, retry, and reconnect rule lives in the
coordinator — the screen only renders the snapshot.

## `engine.gallery`

Gallery sync (hotspot → wifi → HTTP fetch → process → store). Engine orchestrates;
the host renders its own alerts and navigation off `onNotice`.

```ts
engine.gallery.status()                        // progress + queue + glasses gallery summary
const off = engine.gallery.onStatus((s) => { … })
const offNotice = engine.gallery.onNotice((n) => { … }) // user-actionable notices
await engine.gallery.sync(); await engine.gallery.cancel()
await engine.gallery.refreshStatus()           // ask glasses to refresh their summary counts
engine.gallery.removeFilesFromQueue(names); engine.gallery.clearQueue()
```

## `engine.miniapps`

Miniapp lifecycle. The miniapp **WebView is a host component** — engine ships the
bridge primitives (`buildMiniappGlobalsScript`, `buildUiShim`, the JSRuntime
router), exported from `@glassly/engine` directly; your app mounts the `<WebView>` and
wires it to those. This facade is the lifecycle half.

```ts
engine.miniapps.list()                          // installed miniapps (snapshot)
const off = engine.miniapps.onChanged((apps) => { … }) // subscribe; returns unsubscribe
await engine.miniapps.refresh()                 // re-fetch the installed list
await engine.miniapps.start(app, opts?)         // start + foreground a miniapp
await engine.miniapps.stop(packageName)
await engine.miniapps.setForeground(packageName)
engine.miniapps.clearForeground()
await engine.miniapps.stopAll()
await engine.miniapps.install(url, opts?)
await engine.miniapps.uninstall(packageName, version?)
```

## What the host provides

The boundary is small and explicit: the host provides its **UI**, its **login**, and
optional **config** — engine owns the entire runtime. The only required seam is
`auth.getSubjectToken` (passed to `configure()`); engine owns token exchange, refresh,
storage, the cloud client, glasses/BLE, settings, speech, display, and the miniapp
runtime from there.

You may see internal `configureRuntime(...)` wiring in the first-party Glassly App —
that's **transitional** scaffolding for migrating Glassly's own screens, not part of
the OEM contract. It deletes itself as each domain lands in engine. An OEM only ever
calls `configure({auth, config?})` + `start()`.
