Role: Team Leader, Programmer
Platform: Unity, Kinect2.0, HTC Vive
Team size: 3
Project Duration: 04/2018-04/2019
Won the 1st Prize of 12th Chinese Collegiate Computing Competition(Beijing Division) in March 2019 and the 3rd Prize(National Competition) in August 2019.
The Beijing Courtyard program is a role-playing VR game using MS Kinect2.0 and HTC Vive based on Unity platform. It aims to guide people to experience the charm of the currently fading Chinese traditional architecture and culture. For three different levels, we modelled three different Beijing courtyard modes(small, medium and large) using 3ds Max for three game levels,and set up corresponding plots and roles for each of them. Players need to act as different roles and finish three parts: building construction, roaming and tasks.
WHAT I DID:
There are three levels and eleven tasks in total and I developed the second and third level. As leader of the three-student team, 50% technological realization was completed by me.
· Implemented four distinct human-computer interaction modes that allow users to roam in the environment: posture recognition, speech recognition, synchronization, Vive controller interaction
① For posture interaction, I tested the effectiveness of different postures before choosing the most appropriate ones and adding event calling for them;
② The synchronization and the Vive controller interaction aim to enhance players‘ immersive feeling;
③ I also added speech recognition function to create a barrier-free game environment.
【LEVEL ONE】
Role: Poor Scholar (one who passed the imperial examination to be a teacher)
Task 1: Small Courtyard Construction
Rule: According to given map and common sense,select, grab building models from the list and place them to the appropriate place
·Implemented basic functions: Air mouse, gesture recognition, dialog system, timing & scoring
·Developed a function that creates a prefab of the building while clicking on the picture
Task 2: Roaming
·Implemented an interaction mode:Speech Recognition
Audio Command:
*FORWARD/ GO AHEAD * BACK / GO BACK
* TURN LEFT / RIGHT *HELLO / WAVE
* RUN *STOP * JUMP
Task 3: Coin Collecting
Rule: use speech recognition to control avatar to roam in the environment, pick as many coins as possible
【LEVEL TWO】
Role: Merchant
Task 1: Medium Courtyard Construction
Rule: According to tints given by NPC (they talk about the functions and features about the house), select, grab buildings from the list and drag them to the appropriate place
·Dialog system: Used Fungus Plug-in
Task 2: Roaming
·Implemented different interaction modes:
①Posture Recognition：
*ZoomOut: move forward *Zoom In: stop
*LeanLeft: turn left *Lean Right: turn right
②Vive controller interaction：Teleport by Vive controller
Task 3: Yuanbao Collecting(an odd-shaped gold or silver ingot formerly used as money)
Rule: Move to catch as many Yuanbao as possible
·Create Yuanbao randomly and consistent that falls down
Task 4: Calligraphy Imitating
Rule: Imitate and draw strokes of Chinese characters based on template work
* make a fist: start to write *open hand: stop writing
【LEVEL THREE】
Role: General
Task 1: Large Courtyard Construction
According to given poems and names of buildings, select, grab buildings from the list and drag them to appropriate places
·Used Dialog system Plug-in
Task 2: Roaming
·Implemented interaction mode:Posture Recognition
*Jump:move forward *TPose: stop
*Raise Left Hand:turn left(90°) *Raise Right Hand: turn right(90°)
·Implemented a navigation module and a working region definer
Task 3: Arrow Throwing
Rule: Select the board with the right answer to some questions of historical event by throwing arrows to it.
·Implemented function that registers an arrow to participants' hands
Task 4: Chinese Martial Art Learning
Rule: Imitate the motion of models
·Implemented an interaction mode: Synchronization
· A score is calculated based on the degree similarity between the joint angles of the given model and avatar's
Game Environment: