Dataset Construction
=====================

Forza Horizon 5
-------------------

1. Platform Construction
~~~~~~~~~~~~~~~~~~~~~~~~~
The platform is designed to automate gameplay recording, control signal collection, and telemetry extraction.  
It consists of the following components:

- Automated game launching and screen recording using OBS with custom scripts at 2560×1600 resolution to avoid performance degradation.
- Automatic control signal generation, simulating mouse and keyboard operations to drive the game character under randomized behaviors.
- Telemetry data extraction via socket communication, collecting real-time positions (XYZ), velocities, and accelerations.
- Uploading recorded videos and telemetry data to cloud storage in a structured format with metadata (timestamp, biome, etc.).
- Automated stagnation detection:
    - If the player's position remains within a small radius for more than 2 seconds, a backward movement is triggered.
    - If for 40 seconds the trajectory fits within an 80m-radius circle, the system triggers random teleportation.
    - Stagnation detection is performed using telemetry comparison and is synchronized with control signals.

2. Data Collection
~~~~~~~~~~~~~~~~~~~
Data collection aims to acquire diverse gameplay trajectories under randomized control conditions.  
It consists of the following steps:

- Randomized control signal generation covering forward, left, and right movements.
- Mouse and keyboard simulation to navigate across different biomes. 
- Continuous screen recording at native resolution with GUI elements removed using ReshadeEffectShaderToggler.
- Concurrent telemetry logging via in-game API to support collision detection and motion analysis.

3. Data Processing
~~~~~~~~~~~~~~~~~~~
Data processing is conducted to organize and filter the collected samples for high-quality datasets.  
It is divided into the following parts:

- Preprocessing:
    - Clip segmentation: recorded videos are divided into short clips aligned with control and telemetry logs.
    - Clip labeling: each clip is annotated with metadata tags (scene type, motion type, biome).
- Filtering:
    - Class balance adjustment: ensuring balanced distribution across different biome types (across woods, grass, sea, field, river, others (ratio: 12%:15%:18%:16%:15%:9%:15%))
    - Collision detection: Using telemetry acceleration spikes and angular deviations to filter collision events.
    - Stagnation detection: discarding clips where players are stuck or backward motion dominates.
    - Visual artifact detection: removing clips showing large inter-frame pixel differences , sudden lighting/texture glitches.

Cyberpunk 2077
---------------------

1. Platform Construction
~~~~~~~~~~~~~~~~~~~~~~~~~
The platform provides a server-based environment enabling manual gameplay recording with enhanced visual fidelity.  
It consists of the following components:

- Deployment of multiple cloud servers with server image cloning for scalable recording.
- Reshade-based visual enhancement:
    - Lighting, shadow, and texture sharpness improvements.
    - GUI elements are disabled via shader togglers.
- Support for parallel screen recording and precise control signal tracking through socket-based virtual input emulation.
- NPCs and moving vehicles are disabled using mods to reduce scene complexity.

2. Data Collection
~~~~~~~~~~~~~~~~~~~
Data collection is conducted through manual gameplay by designated annotators under strict operational constraints.  
It consists of the following procedures:

- Annotators connect to cloud servers through low-definition, low-latency streams.
- Only a simplified set of operations is allowed:
    - Forward movement (W key).
    - Camera adjustments (U/D/L/R keys for up/down/left/right).
- Simultaneous recording of gameplay video and control signals; Cheat Engine is used to track coordinates for collision checking.

3. Data Processing
~~~~~~~~~~~~~~~~~~~
Data processing ensures the collected Cyberpunk samples meet quality and consistency standards.  
It is divided into the following parts:

- Preprocessing:
    - Clip segmentation: gameplay is divided into small segments synchronized with control logs.
    - Clip labeling: each clip receives metadata such as scene type (urban indoor/outdoor, day/night).
- Filtering:
    - Stagnation detection: removing clips with prolonged scene stasis.
    - Visual artifact removal: excluding footage with lighting glitches, texture corruption, undefined visual artifacts.
    - Collision filtering: based on coordinate jumps detected via Cheat Engine tracking.