Software

• Unity
• Regard3D
• Meshlab
• Cloud compare: https://asmaloney.com/software/
• (Cinema4d)

Scanning your object

• You can go and get a full body scan at a studio.
• You can also get little ones. eg. Einscan pro 2x.
• Another option is GUI scanner - you can buy this really cheap!!
• In this course we used our smartphones. It was important to think about background, light etc.
• Of course you need to get all iof the angles, so do like rings around the object you’re scanning.
• Tilt the camera when you’re going over the top of the object to get the top of it

How many photos should you take?

• Don’t take 500!  250 works. 60 minimum
• aim for a 30% overlap between the images
• Sequence is not important

Process

• Take the photos
• Import into Regard3d
• Click ‘Compute Images’
• You can select show matches and show key points (not essential?)
• Click ok
• Click Triangulation
• There are two engines that this process is done with:
  • Incremental structure from motion
  • and one other one.
  • Try the first one and see if it works. If it doesn’t try the second.
• Now click Create dense point cloud. Use the default settings and click OK.
  • (Level here means how many points it creates)

(Mine took like 1 hour)

• If it’s not really working it could be….
  • Light,
  • Sharpness of the image,
  • How the photos are taken.

(With the photos - I think it’s fine to have a mixture of close up and far away images - someone else in the group seems to have done fairly random images, like not all from the same position, and it looks pretty good.)

• Right click on Densification and click Export point cloud
• Save it as like ‘v1’, ‘from regard3d’ or something so you know what it is. *This will be a .ply*

(An optional alternative(?) to the thing below: You can import the point file into meshlab and you can delete the bits you don’t want. BE CAREFUL IN MESHLAB. Once you do a step you cannot go back. Meshlab is good to clean it up and delete bits that you don’t need. Always work on a copy.)

• Open in Cloud Compare
• Go file > open > click ‘all file types’ so you can open the .ply (or at least I had to do that)
• We now have to change this into a binary file. [in cloudcompare]
• If you have a really big point cloud (which is the number in bottom left property screen) go Edit > Subsample
• Select ‘random’ from dropdown
• Then you choose how many points. 30,000 was suggested for this. Could do 100/150,000.
• Then save (make sure you have selected what you wanna save in the left menu)
• Save it as a .ply
• And then click binary (it comes up with a dialogue box saying ASCII / BINARY)

(Unity only reads binary)

[Fun thing: Export the ASCII file and open it in a text editor: you can see fairly cleanly all of the point coordinates - nice]

Unity

(important bits: The little icons in top left  [QWERTY]. Hand symbol is for moving around the area. Next one, the Move symbol: moving around the object. So if you are on camera (selected on the left) then you will be moving around the camera. Rotate tool rotates objects. Press command and you can move around. Press alt and you can look around)

Down at the bottom left is the Project Folder. Here you can save scenes. If you go to file, save as, you can save the scene. So you can jump through different scenes. Maybe this is more interesting for gamers - you have several scenes and can jump through, you make different environments

In the asset folder you have all of your objects. KEEP IT CLEAN !! There are also packages. You can download packages.

To make objects:

• Go to Game Object in the menu bar at the top
• Click 3d object > Cube
• (You might need to have assets selected in the bottom, I don’t know)

To make a material

• You need to go in the assets folder, right click and click create material
• Then you make the material
• And then you can drag and drop the material onto your cube

Next section...

We will be downloading bits and using links from here: https://github.com/aleannox/soundstack_2019

• First link is to Unity asset. Open that in unity
  • Click download when it’s open in Unity
  • Then click import
  • Import it all

[https://github.com/keijiro/ — POINT CLOUD HERO !!]

• Download Keijiro’s PCX thing (second GitHub link)
• Import this package into unity. Then you have the plugin
• Also Download/ & install LASP (3rd GitHub link)

(Note. Unity works on a project basis so your plugins etc are just in the project. You might want to have a central place(??))

Now, back in Unity:

• Right click in Assets and create a folder.
• Call it Pointclouds.
• Go to the folder of your unity project (in Finder). [If you are in Unity you can ‘reveal in finder’]
• Copy your .ply file into the Unity Pointclouds folder
• Then in Unity when you open your point clouds folder you will see that the file is there! :)

Then...

• Create a new scene
• In the browser go to Standard Assets
• Create First person character 
• Drag it into left panel
• He has his own camera. Change the background to black
• When you’re in the rigid body bit of the right hand side panel you can untick gravity. Click FPS controller on the left to do this
• Above rigid body there is first person controller script. Field called gravity multiplier. Insert 0
• In another bit you need to untick audio listener

(By the way - you need to edit things in scene mode rather than game mode. Changes made in game mode are lost)

• Bring your ply file into the area
• Back here https://github.com/aleannox/soundstack_2019 download that master folder.
• Change that PCX plugin we got earlier — overwite the PlyImporter.cs code with the one from that master folder we just downloaded.
• Also put Audioreactivepointcloud.cs in the script folder in Unity (in Finder)

Making the points react to audio

Basically drag this script onto your point cloud bit in the left section of Unity and when you go in the game and click play it should react to volume.

You can change the code of course.