Creating AR with Unity: Transitioning from Marker-Based to Markerless Experiences
I woke up this morning mad at my coffee table. Besides being barely decorative and mostly non-functional, my coffee table just takes up space in my room. So today I have decided to use it as a table top for my AR experience with Unity’s AR tools. Hopefully you learn a lot more than how to hate your coffee table.
Marker-based Augmented Reality experiences require a static image called a “trigger photo” that a person can scan using an AR enabled device (usually a smartphone but your laptop’s webcam would do just fine) via an augmented reality app. The mobile scan will trigger the AR content (video, animation, 3D or other) to appear on top of the marker. Common applications include: AR cards & educational tools. It’s counterpart, the Markerless Augmented Reality doesn’t require this trigger photo to start the AR experience, just like in AR games like Pokémon GO it works by scanning the environment and locating a suitable surface to begin the AR experience (my dear coffee table).
I’ve come across a couple of articles showing how to create Marker-based or Markerless AR in Unity but I’m yet to find one that shows how to transition your Marker-based AR to Markerless. That’s why I’m here, to guide you through the jungle that is Unity and help you make that transition easily.
Setting up the scene
First we start by creating a Unity hub account and installing Unity if you haven’t already. When unity is opened for the first time, you will need to install an editor. Go to the Installs tab and select Install Editor.
Install the recommended editor (Unity 6000.0.28f1 LTS in my case) Make sure to check the Android Build Support and its dependencies (OpenJDK and Android SDK & NDK Tools) and then click Continue. The installation should take around 10–20 minutes.
After the installation is complete navigate to the Projects tab in the vertical left menu and on the top right click New project. Select Universal 3D (core), name your project whatever you feel like and select Create Project. Your project prepares for first initialization and you should be good to go in 2 minutes.
To start our AR Experience, we’ll make use of Vuforia, an augmented reality software development kit (SDK) for mobile devices that enables the creation of augmented reality applications. Vuforia supports both approaches (Marker-Based and Markerless), and you can set up a project that combines marker-based functionality (using Trigger Photos) and markerless functionality (using Ground Plane Detection).
Installing Vuforia Engine
To start this step, download the Vuforia package for Unity. Please note that you cannot download the package unless you have registered yourself as a Vuforia Developer.
Next, go to your downloads and double-click on the downloaded Unity package and you will see a pop-up in the Unity Editor telling you to import the Unity package. Click Import.
To check if Vuforia is installed, right-click in the Hierarchy panel (by the left). If you see ‘Vuforia Engine’ in the dropdown menu, you’re good to go.
In the Hierarchy panel, delete “Main Camera” from the scene. Then, right-click Hierarchy panel > Vuforia Engine > Add AR Camera to the scene.
In your Project Window, head to Assets > Resources and open up VuforiaConfiguration.
Now you’ll need a license key — just go to the Vuforia Developer Portal > My Account > Plan and Licenses and click “Generate Basic License” Add a name to your license and click confirm. (you must be logged in for this step to work — obviously).
On the license table, select the newly created license and tap the license key to copy it to clipboard and paste it into the “App License Key” field in Unity.
The Basic Plan which we are currently using lets us do pretty much everything — make unlimited apps using features like Image Targets, Multi Targets, Cylinder Targets, VuMarks, and Ground Plane. And don’t worry, our apps won’t have any Vuforia watermark on them.
Marker-Based: Creating Trigger Photo
To create a trigger photo, Go to the Vuforia Developer Portal > My Account > Target manager > Generate Database. Create a Database name (note that Database Name must have no spaces and can only contain letters, numbers, dashes and underscore) and leave the radio button checked on “Device”. Click on Generate.
On the Database Table, select the newly created Database after it opens, click on Add Target.
On the pop-up dialog box, select Image as Type and upload your marker/ Trigger Photo. It is important to make sure that your Image is very clear. You can use images of items from your environment as your trigger photo. I am using an image of the Celebrations chocolate TUB because the physical object is sitting on my work table (don’t judge me).
Name your trigger photo as required, usually it automatically takes the name of the file uploaded but you can change it to something you’re more happy with. Click on Add.
Your image will show up in the database with a star rating next to it. More stars means your camera will have an easier time detecting and tracking the image. By the left side of the Database, select the image using the checkbox and select Download Database. On the pop-up, select Unity Editor as the Development Platform and select Download.
Double click the downloaded file to add it to Unity by selecting Import when prompted. After importing your database, open VuforiaConfiguration again (it’s under Assets > Resources). Look for your database “HelloVuforia” in the Databases section. If you see it there, you’re all set — the import worked.
Render 3D content on Trigger Photo
To show a 3D object over the trigger photo, we need to add an Image target to the scene. Right-click in Hierarchy Panel > Vuforia Engine and add an Image Target to your scene.
While the Image Target is selected in the Hierarchy Panel, on the inspector panel, look for “Image Target Behaviour (Script)”. 3 quick things; Set Type to From Database, set Database to your database name and finally set Image Target to the name you used for the trigger photo. You should see the Image Target update in the scene.
To add a 3D object over the Image Target, we need to get 3D assets. Download any free 3D assets from the web and import it to your Project folder > Assets by simply dragging and dropping the content. I got some free models from the Unity Asset Store.
Select the model you want to use and drag it to the Hierarchy Panel > Image Target. This will make the model a child of the Image Target. Adjust the position and scale as needed to sit directly above the Image Target.
Adjust the position of the model as necessary.
To test the Marker-Based AR, hit the play button to test it out with your webcam. It should work without issues. (we only need to know it works, it is not necessary to load on a mobile device yet).
Transition to Markerless AR
To begin, right click on the Hierarchy Panel > Vuforia Engine > Ground Plane > Ground Plane Stage. Transfer the 3D model from Image Target to the Ground Plane Stage. Delete Image Target, you should be left with this
Next, Rightclick in the Hierarchy Panel and select Vuforia Engine > Ground Plane > Plane Finder.
Drag the Ground Plane Stage object from the Hierarchy Panel into the Anchor Stage parameter of the Content Positioning Behaviour component in the Plane Finder Hierarchy Object.
Testing the Markerless AR
To test the Markerless AR, we’ll be running the simulation on Mobile. I use an Android so I will be building on Android platform. Go to File > Build Profiles.
By default it’s on Web, now go to the Android tab and switch platform. This should take some time since it’s your first time using the Android platform and you might need to install an Android module with Unity Hub.
After that is complete, click on build. Save the .apk file in a folder you will remember with a name you will remember. Note that the first time you build, it might take up to 15–20 minutes (maybe shorter, my laptop is not the best).
After build is complete we need to transfer the .apk file to an android device for it to be run. I have already connected my android device to my laptop so I will simply drag and drop the .apk file into downloads. You might decide to mail it or use Google drive/ Dropbox — whatever works for you.
On my phone I check downloads and install the application. If you’re new to installing applications that are not from the Google Playstore, you might need to “allow unknown apps to be installed” or something along those lines. After installing the App, you will also need to allow camera access to the app after then we can test our app. Everything should work fine as Shown
It should go without saying that this is just for learning purposes and that you have freedom of expression in creating your own AR experiences, the possibilities with Unity and Vuforia are endless. Try it out, push the boundaries of creativity, and share your unique AR projects with the world. Have questions or exciting ideas? Let’s keep the conversation going — leave a comment or connect with me on X (formerly twitter). I have attained great inner peace now that I have put my coffee table to good use, see you next time.
