Mobile E-Learning · AI · UX Design · Learning Experience Design
Overview
Select a phase in the diagram to jump to that section.
This project follows the ADDIE framework. I began by analyzing the current English-learning challenges and the specific learning characteristics of EFL learners aged 6 to 12. Based on these learning needs and identified problems, we designedthe application's structure, features, and prototype. We then developed the app prototype using Gemini and Codex. So far, we have tested it in class and with five young users. In the next phase, we plan to implement and evaluate the app with a larger group of target users.
Analyzing current English-learning challenges and the specific learning characteristics of EFL learners aged 6 to 12.
The target audience (elementary school students) belongs to Generation Alpha, a generation with a reduced reliance on text and a significantly enhanced preference for visual content. Fundamental EFL learning for this age group requires extensive vocabulary acquisition, contextualized reading, and listening practice. Embodied learning modalities such as handwriting, gesturing, or physical interactions like photography can also facilitate the connection between visual and motor networks, thereby enhancing memory retention.
When EFL learners study English words without contextual clues, the mechanism for deep memory encoding is not triggered. Consequently, learners find it difficult to utilize the foreign language in real-life scenarios.
Although digital resources offer flexibility, the interaction is often limited to mechanical “clicking and watching.” This passive consumption leads to boredom and a rapid decay in motivation.
Additional resources, such as picture books or after-school tutoring, often cause extra costs for families. The affordability of English-learning materials remains a significant barrier.
The target audience is children aged 6 to 12, elementary school students learning English as a Foreign Language.
Multimedia learning theory systematically explains how human beings process multimodal information and how design can reduce cognitive load. Meaningful learning occurs as learners repeatedly select and organize relevant information, retrieve prior knowledge from long-term memory, and integrate it with new information.
While multimedia theory guides how we present information, theories of motivation, interest, and engagement explain why a learner chooses to participate and persist. We use Self-Determination Theory to support autonomy, competence, and relatedness, and the Four-Phase Model to understand how interest develops through interaction with the environment.
Creating a complete path from perceptual experience to meaning construction, and then to application and reinforcement.
Make children's daily lives the source of content by using the objects, interests, and physical environments they choose.
Embed photographed objects into a coherent story so vocabulary can be understood in use rather than presented in isolation.
Use exploration, instant AI feedback, autonomous task selection, and collectible learning achievements to support long-term engagement.
To create a contextualized and engaging learning tool, we propose a Photo-to-Story Pipeline. This workflow encourages children to use mobile phone cameras to detect surrounding objects and convert them into digital realia. Once an object is captured and learned through a digital word card, the system advances to a narrative phase and integrates the learner's personal physical reality into the digital learning material.
SnapStory divides the learning process into four stages, forming a complete path from perceptual experience to meaning construction, and then to application and reinforcement.
The user flow organizes the children's learning experience from profile and task selection to camera exploration, story creation, interactive reading, and post-reading review.
Automating the learning process from photos to vocabulary cards and personalized picture books.
This system relies on model services provided by Google AI Studio and uses tools such as the Camera Input module, AI object recognition, vocabulary card generator, AI co-creation engine, and Storybook Renderer to automate the process from photos to picture books.
The application runs on mobile devices with camera capabilities, such as phones or tablets, while on computers, photos need to be manually uploaded to enter the learning process.
01 · Exploration input
Children begin with real-world situations, using a camera to record everyday objects and making the real environment personalized learning material. The system instantly recognizes images and generates vocabulary cards containing pictures, spelling, definitions, and pronunciation, providing multi-channel, multimodal basic input and supporting self-monitoring and repetition practice.
Embodied and grounded learningPhotography creates an authentic connection between English vocabulary and the physical world, transferring learning from floating cognitive concepts to a grounded understanding within a real-world context.
Pre-training + multimediaBefore children begin reading, the camera scavenger hunt introduces target words through a real-life photograph, the word's spelling, and audio pronunciation. When the words later appear in the story, children already have prior experience with them.
Autonomy + triggered situational interestThe mission provides a topic direction but leaves the specific objects up to the user. Tap to Snap and instant AI recognition create a direct link between the real world and the screen, making the experience feel novel and fun.
02 · Story generation
After vocabulary collection, the system automatically generates coherent English stories based on the photos or collaborates with children to brainstorm plots, embedding the exploration results into a narrative structure. This stage transforms memorized material into applicable context and prompts children to understand the meaning and function of vocabulary within a context.
Contextualized vocabulary learningChildren photograph real-world objects and then encounter these words again in AI-generated storybooks. This process provides multiple contextual cues, allowing the meaning of words to be understood in use rather than presented in isolation.
Autonomy + emerging individual interestUsers can choose Magic Story or Co-Writer. In Co-Writer mode, children write a few lines first and AI continues the story based on their ideas, allowing them to express themselves and bring their imagination into English learning.
Coherence + segmentingSnapStory divides stories and interactive activities into manageable segments based on the user's age and English proficiency. A picture book typically has around five pages, preventing children from seeing too much content at once.
03 · Interactive storybook
The story is presented as an illustrated storybook with natural TTS narration and inline translation. Interactive tools such as highlighting, vocabulary cards, word-finding, and spelling tasks reinforce form–meaning–use connections, while short quizzes provide immediate feedback for consolidation.
Signaling + temporal contiguityDuring story reading, the current word is highlighted synchronously when the TTS is reading, allowing children to correspond the sound with the text and providing a clear focus for the learning task.
Spatial contiguity + coherenceVocabulary explanations and Chinese translations appear directly on the same page beside the corresponding text. The reading interface also hides navigation and components unrelated to the current task.
Competence through layered practiceThe same words appear in vocabulary cards, Find the Word, Spell It, reading aloud, and the end-of-story quiz, providing multiple scaffolded opportunities for users to succeed with the same vocabulary.
04 · Post-reading extension
All vocabulary words that appear in the stories are automatically added to a sticker album and personal vocabulary library, forming a visual learning record. The Gym module, based on spaced repetition theory, supports the continuous reinforcement of long-term memory through activities such as flashcards and pronunciation comparisons.
CompetenceExplorer Passport displays the number of stickers and badges the user has collected, allowing children to see how many vocabulary items they have learned and which achievements they have unlocked.
Maintained situational interestThe same set of vocabulary words reappears across different contexts and activities. This keeps practice varied rather than repetitive while strengthening understanding and long-term memory.
Well-developed individual interestLearners can keep photographing new objects, generating non-repetitive stories, and growing a personal vocabulary album. Word Gym also offers a structured way to practice words again and again, supporting a more self-directed learning habit.
Using early implementation and formative evaluation to examine the complete SnapStory learning experience.
Testing context
We tested the prototype in class to review the end-to-end interaction flow and the alignment between the product functions and the intended learning experience.
Participants
The formative sessions followed the learning path from mission selection and photo capture to vocabulary learning, story generation, interactive reading, and review.
Current status
This phase provides an initial implementation and evaluation step. The prototype still requires broader testing before we can make claims about learning outcomes.
This design was my final project for MSTU 4083: Instructional Design of Education Technology and earned an A+.
Through this project, I learned how to move through a complete instructional design process—from analyzing an authentic learning challenge and learners' needs, to defining learning goals and strategies, designing the application structure and learning flow, and translating those decisions into a visual and functional prototype.
I gained hands-on experience applying the ADDIE Model, the Cognitive Theory of Multimedia Learning, and frameworks for motivation, interest, and engagement to guide design decisions rather than adding technology for its own sake. Using Figma for UI/UX design and Gemini and Codex for AI-assisted development, I connected the Gemini API to support object recognition, audio, and storybook image generation.
Most importantly, I learned that an effective AI learning experience comes from aligning every feature with a clear learner need, learning objective, and evidence-based learning principle.
