Localizing an object

Building a world by hand means typing where everything sits. That is fine for a scene you invented, and hopeless for a bench you did not: the plate reader is where the plate reader is, to the millimetre, and no amount of careful measuring gets you there.

Localizing solves that the other way round. You put printed tags on the object, show it to a wrist camera from a few angles, and the platform works out where it actually is.

What you need on the object

Printed AprilTags, of the apriltag_36h11 family. It is the only family supported, and an object record naming any other is rejected outright rather than quietly failing to see anything.

The tag's printed edge length matters, and the default is 20 mm. Whatever your tags actually measure has to be what the object's record says, because the size is what turns pixels into millimetres. Print at 25 mm and tell the system 20 mm and the object comes out a fifth too close.

Where the tags sit has to be recorded too, so the solver knows what it is looking at. That comes with the object. There is no tool for measuring a fresh set of tags onto an object yourself, so an object either arrives ready to localize or it does not.

Tagged units

A tag is a sticker on one physical object. Two copies of the same part have the same mesh, the same shape, and the same grasp points, but they wear different tag numbers.

A tagged unit is one physical copy's set of tags. The object's shared record carries everything that is true of the type; each unit adds only its own tags. That is what lets a lab own three of something without keeping three near-identical copies of its model.

You meet tagged units in two places:

• The Tagged unit picker in the World Builder, which only appears when a type has more than one.
• Tagged units in the Object Database, on the object's card and in its detail view. Objects that carry no separate units show nothing.

Most objects today still record their tags the older way, as part of the shared record, and show none of this. If you never see a Tagged unit picker, that is why, and nothing is wrong.

Working out which copy it is

When a type has several units and you did not pick one, the platform decides from the tag numbers it can see. It refuses rather than guesses: if the tags in view are ones that several units share, it says so and asks you to move to an angle that shows a tag only one of them has.

It also checks the other way round. Once the numbers point at a unit, it verifies the tags actually sit where that unit's record says they do. If they do not, it tells you, and the usual cause is a tag that has come loose or been stuck back on crooked.

This only happens when nothing has already told it which unit to use. Picking one in Tagged unit, or re-localizing an object that already records one, binds it outright and skips identification. So an object with a unit on record will not tell you the hardware has been swapped: it will trust the record. If you suspect a swap, place the object fresh rather than re-localizing it.

Doing it

From the World Builder, which has the full screen reference. In outline:

1. Pick the object type, and its tagged unit if you are asked for one.
2. Add + Localize for something new, or the Re-localize action on a row for something already in the scene.
3. The arms go into freedrive. Hand-guide a camera arm to a viewpoint.
4. Snaps get taken, either automatically when you hold the arm still or by pressing Take Snap. Move to a new angle and repeat.
5. Localize solves from all the snaps together.
6. Check the result, then Place.

Getting a good solve

• Angles beat repeats. Four snaps from four genuinely different directions are worth far more than twenty from the same place. The solver is fusing them into one answer, and two views from the same spot tell it almost the same thing twice.
• Watch the tag counts. Each snap in the list reports how many tags it saw and how many of those belong to the object. A snap contributing no tags on the object is not helping.
• Read the numbers, then look at the picture. Pos σ and Rot σ say how sure the solve is of itself, which is not the same as being right. Show overlay draws the solved model onto the actual photograph. If the outline sits on the object, it worked. If it sits next to the object, a confident number was still wrong.
• The first solve is slow. Later ones in the same session are much quicker. This is expected, so give it a minute rather than pressing things again.

What Place actually changes

Place puts the object into the live world, in both the simulated and the real scene. That is enough for a run, and it is not enough to survive a restart.

Re-localizing changes the object's pose and nothing else. Its name, its index, and its anchors are untouched, which is what makes it safe to correct a placement on an object that skills already reference.

Doing it from a skill

A skill can refine an object's pose mid-run with localize_object_tags, which captures from viewpoints, solves, and writes the refined pose back only if it passes acceptance checks. It is the unattended counterpart to this screen, and it is marked experimental.

The collection parameter there is the tagged unit, for an object type that has more than one.

Related