Get your guests to read the with the power of augmented reality.

Last year, The Verge published a story about a brilliant application for augmented reality — AR house manuals for Airbnb.

Isil Uzum, a designer in Italy, had created a concept design for an AR app that would help guests find their way around a new home using virtual sticky notes attached to stuff around the house. It struck a chord. The post went viral and it seemed like the first “aha” moment for AR on mobile phones.

The only problem was, there’s no way to build this with existing tools like and ARCore. (I explain why further below)

As a team of computer vision experts, this really set our wheels in motion and after a year of development we’ve finally solved the problem. The result of our work is an SDK that lets developers build persistent AR overlays on any physical space (like an Airbnb) and we call it Placenote SDK.

This post will teach you how to build an AR overlay for an Airbnb (or any physical space) yourself, with ARKit and SDK.

By — Isil Uzum

Why can’t this be built with ARKit alone?

The problem with ARKit is that while it detects visual features in the environment to track the phone’s motion, it has no ability to “remember” features in the environment. This is because the point features it detects are not distinct enough to be repeatably found in the same place every time it observes an area.

That means each time an app is opened, ARKit sets an arbitrary (0,0,0) point at the phone’s current position and places all content in the scene relative to that point.

In order to overlay digital content at specific areas in a house, we need a way to persistently fix the positions of AR content in a physical space, so that every time an app is opened, the origin point of your scene snaps to its correct absolute position in the real .

That’s where Placenote SDK comes in.

How does Placenote SDK work?

Placenote SDK integrates with ARKit and enables -based mapping and visual positioning for AR apps built in Unity or SceneKit. Essentially, it lets you map any physical space to create a persistent canvas for your augmented reality content.

Let’s look at how it works.

1. Mapping an Area

Placenote SDK uses a custom feature detection algorithm that looks for “” features in the environment that it thinks can be found persistently across AR sessions. This feature detection is independent of ARKit feature tracking and runs in a parallel thread as you move your phone around.

The collection of smart features forms a 3D feature map of the space that can be saved and uploaded to the Placenote Cloud. Once the map is uploaded, Placenote returns a MapID that can be used to reload or share a map with multiple users.

Here’s a simple Swift code block that shows you how to start a mapping session and save the map when you’re done.

// Start mapping session
// Scan the area with your phone to create map...
// Stop mapping session and save map
LibPlacenote.instance.saveMap(savedCb: { (mapID: String?) -> Void in
      LibPlacenote.instance.stopSession()  },

uploadProgressCb: {(completed: Bool, faulted: Bool, percentage: Float) -> Void in

2. Adding Content

Content can be added to a scene anytime during or after mapping. We designed Placenote to seamlessly integrate into any existing SceneKit or Unity project so there are no changes to the way you create scene objects.

However, remember that when building persistent AR content, you will need to save your content somewhere. If you’re not already saving user generated scenes, you’ll need to save them either to your backend or locally on the device.

Here’s a sample interface we designed in our experiments with building an Airbnb house manual.

3. Reloading a Scene

Once a map is saved, you can load it up whenever you want to open a previously created AR scene in that area. When Placenote SDK is initialized with the loadMap() function followed by startSession(), it downloads the map (unless it’s cached locally) using the given MapID and attempts to match smart features currently detected by the camera, with features in the pre-constructed point cloud map. When it finds a match, it shifts the position of the AR camera to realign the world origin point with the original map.

Swift code block:

LibPlacenote.instance.loadMap(mapId: "<Map ID Here>",
downloadProgressCb: {(completed: Bool, faulted: Bool, percentage: Float) -> Void in
      if (completed) {


To a user, this creates the magical effect of AR content being permanently attached to a point in the real world!

Try it yourself..

If you want to try building an Airbnb house manual app yourself, download Placenote SDK, get a free API key and try out our sample apps for Unity or Native SceneKit.

We’ve built a simple app that lets you save random shapes in a physical space with Placenote. Feel free to fork the repos and modify the samples to create a “Sticky Notes” interface yourself!

Here’s where you can download the SDK. The README has instructions on getting an API key.

Placenote for Unity

Placenote for iOS SceneKit (Swift)

Got a question? Get in Touch with Us!

If you want to build amazing AR experiences on iOS or Unity, partner with us, or join our team, let’s connect! We’re building an SDK for persistent, shared augmented reality experiences and we can’t do it alone 🙂

Learn more at: Placenote SDK.

Contact me at:

Apps you can build with Placenote.

To help you better understand the concept of persistence and perhaps inspire some ideas for persistent AR apps, I’ve curated a wish list of apps I hope might exist in the world some day. If you’re working on something cool in any of these areas, I’d love to hear from you!

  1. AR Drawing app that lets you create and save virtual graffiti in precise locations indoors and outdoors.
  2. Product manuals for office printers, coffee makers etc.
  3. Furniture app that lets you save and share 3D furniture designs in your room, perhaps for interior designers?
  4. Multiplayer game that synchronizes and aligns the viewpoints of 2–3 users in a single location.
  5. Construction documentation app with an AR interface to find highlighted points of interest on a construction site.
  6. equipment manual for operating complex machinery.
  7. Guided museum tour in AR.
  8. Product search and navigation in a store
  9. And many more ideas to come….!

Source link
thanks you RSS link


Please enter your comment!
Please enter your name here