Harbin ARcollage

An app for tourists in Harbin, China to interact with historic architecture and culture through AR technology, has been recognized by international design awards.

· Mobile AR · Cultural Experience · Spatial Storytelling · Digital Heritage ·

Roles

UX & AR interaction designer

Timeline

4 months

August - December 2024

Team

1 designer

Skills

User Research, Competitive Analysis, Wireframing, Prototyping, Usability Test

Overview

Designed an AR mobile experience that enables visitors to explore historic architecture in Harbin, China, through interactive storytelling and spatial interaction.

The project explores how emerging technologies can enhance cultural engagement beyond passive sightseeing.

The Challenge

1️⃣ Commercialization is overshadowing the original cultural and historic identity of Central Avenue in Harbin, China.

2️⃣ Limited on-site information makes it difficult for visitors to understand the context behind heritage buildings.

3️⃣ Existing experiences are passive, lacking meaningful interaction and engagement.

The Solution

—— Full Case Study ——

01 Research

On-site information is fragmented, losing authenticity and lacks narrative structure.

First, the unique cultural vibe is gone with commercialization.

Historic buildings have been transformed into shops from inside out, squeezing out those ones which actually well represent the local culture and transforming a unique cultural avenue to an ordinary business street that can be seen anywhere.

Cultural Street that Keeps the Style from the 1940s

Overcommercialized Avenue Today

Tourists and locals often complain about this on social media.

In the Harbin Municipal Tourism Bureau's satisfaction survey about the Central Avenue,

more than 70% of tourists said they hope to see more contents related to Harbin's history and culture, rather than excessive commercialization.

Second, limited on-site information makes it challenging for visitors to learn about the Central Avenue history.

The boards on the buildings only have basic text information, like name, construction time, historical function, and style.

But visitors also hope to see stories, details, changes and images through the decades.

Black boards hung on heritage buildings with only basic info

Roads and streets cross with each other, creating a district which is hard for visitors to locate where they are.

There is only 1 map about heritage buildings on site to give an overview.

Can’t help visitors when getting lost in the district with so many crossings

02 Understand the Visitors

Visitors seek systematic information and deeper cultural experience beyond surface-level sightseeing.

Based on the research, I concluded 2 visitor groups in user persona to better understand user needs and pain points.

03 Design Goals

I summarized a “How Might We“ and 3 design goals:

How Might We create an engaging and interactive experience for people to learn, play and enjoy?

My design goals are:

🎯 1. to provide systematic and authentic information about buildings on Central Avenue;

🎯 2. to provide mapping service that can help visitors locate where they are while they are visiting;

🎯 3. to enable more interactions between visitors and the place, making their trip more enjoyable and impressive through gamification.

04 Brainstorming

With clear design goals, I brainstormed 3 ideas and scored them 1 - 5 in the following criteria:

Interaction: how much visitors can interact with the place through it;
Information Amount: how informative it is;
Accessibility: how easily visitors can access the entry to it;
On-site Experience: how satisfied visitors are when using it on site;
Off-site Experience: how satisfied visitors are when using it off site.

Based on the scoring, I decided to move forward with the “AR Mobile App”.

Competitive Analysis

In order to understand AR technology’s limitation and application, I conducted competitive analysis of 6 products:

All have a map service and detailed infomation page for users to locate and learn;
Interactive & gamified festures, like 3D scene and puzzle game, can significantly increase engagement.

Experience Framework

To summarize, I will focus on designing the following 3 user flows.

05 Prototyping

Usability Test & Iterations

I invited 6 users to do the test and iterated my design.

🧵 Iteration 01

Make every image in “Visual History” more accessible.

🧵 Iteration 02

Give users instructions of some interactions in the AR Scan feature.

06 Delivery

Map Service

🎯 Fitting Design Goal 2 to provide mapping service that can help visitors locate where they are while visiting the place.

Learn More

🎯 Fitting Design Goal 1 to provide systematic and authenticinformation about buildings on Central Avenue.

The concept of visual design is inspired by “Museum”,

giving users the experience just like wandering in an architectural art museum.

AR Scan

🎯 Fitting Design Goal 3 to enable more interactions between visitors and the place.

Collection Box & AR Create

🎯 Fitting Design Goal 3 to enable more interactions between visitors and the place, making their trip more enjoyable and impressive through gamification.

07 Reflection

This project explores how emerging technologies like AR can reshape cultural experiences, shifting them from passive observation to interactive engagement.

Rather than simply adding digital layers to physical spaces, meaningful experiences require a clear narrative structure and intentional interaction design. Design plays a critical role in shaping how technology is experienced.

08 Next Steps

⛳ Precise digital 3D models

The precision of 3D models which the project use is crucial to the visual effect of the app. If time allows, I would like to try drone photography to acquire more photos of buildings as detailed as possible in order to produce nice building models.

⛳ Interactive AR gamification

City collage is not the only way to use generated 3D models. Given more time, I would like to provide more ways for users to choose, such as taking selfies with heritage buildings, to make this app stand out and improve user adoption.

Appendix:Technical Considerations

📌 Generate 3D models

Autodesk Recap Photo: I took photos of heritage buildings on Central Avenue and generated digital 3D models through photogrammetry. The key point is to ensure every photo has 70% overlap with each other.

📌 AR scan to show 3D models

Unity+Vuforia AR : to simulate the process of AR scanning a building facade & show digital 3D models.

I simplified the facade structure into black and white targets for Vuforia AR.