Cameras
Capture a posed still or a video from a wrist camera or another camera, and relocalize a tagged object from a wrist-camera image.
A camera is named by the arm it is mounted on ("left", "right"; "left_arm" and "right_arm" are accepted) or by the name of another camera the rig serves. list_cameras() returns the names available now.
The camera functions never raise on a camera failure: they log it and return None. Check the result before using it. A cancelled run still raises ExecutionCancelledInSkillError at their pause point.
A still image
capture_image(camera=None, name=None, *, depth=True, quality="normal", profile=None, lens_position=None,
export=False, save_to_project=None, arm=None, capture_name=None)Takes one image from camera and saves it under the folder name name. Returns a Capture, or None on failure.
depth=Falsesaves colour only.quality="best"switches the camera to its highest-quality mode for this one frame and back afterwards. Other readers of the camera see the switch.profile=switches to one of the camera's advertised profiles for this one frame and back afterwards (it takes precedence overquality).lens_position(0-255, needsprofile) holds the focus there for that frame, for a target nearer than the calibrated focus.arm=andcapture_name=are accepted forcameraandname;save_to_projectis ignored.export=Truealso uploads the capture to your lab's own storage bucket, when data export is configured on the rig.
Every capture is also written into the project at data/captures/<execution_id>/<name>/, which syncs with the project.
Capture field | Value |
|---|---|
color | Path of the saved color.png, or None. |
depth | Path of the saved depth.npy, or None. |
project_dir | The capture's folder in the project, or None when no project is bound. |
world_P_cam | The camera's pose in the world when the frame was taken (Pose), or None for a camera the world model does not place. |
stamp | When the frame was exposed (s). |
camera, name | The camera and the capture name. |
A Capture is also a string: the path of the capture folder. It can be passed straight to ask_user_slack(image=...), which sends every image in the folder. send_slack(files=...) takes files, so pass cap.color.
cap = capture_image("left", "deck_check")
if cap is None:
raise RuntimeError("deck_check: the camera returned no image")
choice = ask_user_slack("Is the deck clear?", ["Yes", "No"], image=cap)Video
rec = capture_video(cameras, name, seconds, *, fps=None, depth=False, export=False)
with record_video(cameras, name, *, fps=None, depth=False, export=False) as rec:
move_arm(...)capture_video records for seconds and returns when done. record_video records while the block runs, and stops at the block's end, on an exception in it, or when the run is cancelled. cameras is one camera name or a list, recorded in sync into one folder. A pause does not stop a recording.
fps=Nonekeeps every frame the camera delivers; a lowerfpskeeps fewer.depth=Truealso saves the full image and depth of every kept frame.
The recording folder holds recording.json and one subfolder per camera with video.mp4 and frames.jsonl (one line per frame: time, camera pose, arm joints). A camera that drops out mid-recording leaves a gap that is recorded in both files; recording resumes if it comes back.
Recording field | Value |
|---|---|
dir | The recording folder. |
cameras | The cameras recorded (any that could not be opened are left out). |
videos | Per camera, its video.mp4, or None if it delivered no frame. |
frames, fps | Per camera, frames written and the rate. |
gaps | Per camera, its outages. |
Both return None (and record_video still runs its block) when no camera could be opened.
Relocalizing a tagged object
localize_object_tags(object_name, viewpoints=None, *, arm=None, use_prior=True, min_detections=2,
pos_sigma_gate_mm=15.0, max_move_mm=None, ...)The options shown are the common ones; the cheat sheet lists the full signature.
For an object that carries AprilTags: takes one image with the wrist camera of arm where the arm is now, solves the object's pose from the tags it sees, and writes the refined pose into the world model only if the result passes its checks. It never raises; it returns a dict.
use_prior=Truestarts from the object's current pose in the world and treats the result as a correction of it.Falsesolves from scratch, for an object that may have moved far.- The result is accepted only when it is valid, saw at least
min_detectionstag observations, and its position uncertainty is withinpos_sigma_gate_mm.max_move_mmalso rejects a result that moves the object further than that.
| Key | Value |
|---|---|
success | True when the pose was written. |
moved_mm | How far the object moved (on success). |
before, after | The pose before and after, as {"xyz", "wxyz"}. |
reason, gate | Why it failed or which check rejected it. |
A result that passes or fails the checks is recorded in the run log. Move the arm so the camera sees the tags before calling it. See Localizing an object for tagged objects.
res = localize_object_tags(plate_id, arm="left", max_move_mm=20.0)
if not res["success"]:
raise RuntimeError(f"could not localize the plate: {res.get('reason')} {res.get('gate', '')}")Updated 1 day ago