A Blog!

AKCIT S2R Simulator Investigation

3 min read

Goals:

  • Run simulator
  • Get images

Important:

Rendering Landmark Images

Downloading the dataset

After you request access to the Matterport3D team, they will give you the download link for the download_mp.py script

# set one scene to download
echo i5noydFURQK > scans.txt
 
# These are used in the default simulator tests so you should probably add them so you don't need to worry about errors or altering files (I haven't had the time to rewrite this "guide" yet sorry)
# 17DRP5sb8fy
# 1LXtFkjw3qL
# 1pXnuDYAj8r
 
 
python download_mp.py -o base_dir --scans scans.txt --type object_segmentations --task_data semantic_voxel_label_data semantic_voxel_label_models

How to get the simulator running

# Get simulator
# README: https://github.com/peteanderson80/Matterport3DSimulator?tab=readme-ov-file#building-using-docker
 
# Make sure to clone with --recursive
git clone --recursive https://github.com/peteanderson80/Matterport3DSimulator.git
 
cd Matterport3DSimulator
 
docker build -t mattersim:9.2-devel-ubuntu18.04 .
 
# export MATTERPORT_DATA_DIR=<PATH_TO_SCANS_FOLDER>
export MATTERPORT_DATA_DIR=/home/gus/ufg/AKCIT/Matterport3D-Dataset-Downloader/base_dir/v1/scans
 
docker run --gpus all -it --mount type=bind,source=$MATTERPORT_DATA_DIR,target=/root/mount/Matterport3DSimulator/data/v1/scans --volume `pwd`:/root/mount/Matterport3DSimulator mattersim:9.2-devel-ubuntu18.04

Now (from inside the docker container), build the simulator code:

cd /root/mount/Matterport3DSimulator
mkdir build && cd build
cmake -DEGL_RENDERING=ON ..
make
cd ../
 
# edit connectivity/scans.txt to only the scans you have downloaded
 
# precompute stuff for efficiency
./scripts/downsize_skybox.py
 
# if you need depth images (takes a while)
./scripts/depth_to_skybox.py
 
 
# This should complete with no errors if you donwloaded the entire dataset!
# If you only donwloaded a few scans it will break! It's probably fine...
# (The file src/test/main.cpp on line 49 shows why it breaks)
./build/tests ~Timing
 
# This one should work regardless however.
./build/tests Timing
 
 
exit

Interactive demo:

xhost +
 
 
# didn't work:
# docker run --gpus all -it -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --mount type=bind,source=$MATTERPORT_DATA_DIR,target=/root/mount/Matterport3DSimulator/data/v1/scans,readonly --volume `pwd`:/root/mount/Matterport3DSimulator mattersim:9.2-devel-ubuntu18.04
 
 
# render works but you can't move the camera
# docker run --gpus all -it -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --mount type=bind,source=$MATTERPORT_DATA_DIR,target=/root/mount/Matterport3DSimulator/data/v1/scans,readonly --volume `pwd`:/root/mount/Matterport3DSimulator -e="QT_X11_NO_MITSHM=1" mattersim:9.2-devel-ubuntu18.04
 
 
# works
docker run --gpus all -it -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix --mount type=bind,source=$MATTERPORT_DATA_DIR,target=/root/mount/Matterport3DSimulator/data/v1/scans,readonly --volume `pwd`:/root/mount/Matterport3DSimulator -e="QT_X11_NO_MITSHM=1" --device=/dev/input mattersim:9.2-devel-ubuntu18.04
 
 
 
cd /root/mount/Matterport3DSimulator
 
# edit src/driver/driver.py and change line 24 to a scan you have downloaded
# eg change sim.newRandomEpisode(['1LXtFkjw3qL']) to sim.newRandomEpisode(['i5noydFURQK'])
 
# Set sim.setDepthEnabled(False) to True if you ran ./scripts/depth_to_skybox.py
 
 
python3 src/driver/driver.py

nvidia-docker to toolkit conversion

nvidia-docker is deprecated, you need to replace it with the NVIDIA Container Toolkit. Basically you just need to swap nvidia-docker run with docker run --gpus all

The command

nvidia-docker run -it --mount type=bind,source=$MATTERPORT_DATA_DIR,target=/root/mount/Matterport3DSimulator/data/v1/scans --volume `pwd`:/root/mount/Matterport3DSimulator mattersim:9.2-devel-ubuntu18.04

Becomes:

docker run --gpus all -it --mount type=bind,source=$MATTERPORT_DATA_DIR,target=/root/mount/Matterport3DSimulator/data/v1/scans --volume `pwd`:/root/mount/Matterport3DSimulator mattersim:9.2-devel-ubuntu18.04

nvidia container toolkit

github nvidia docs tutorial

Backlinks