Topic: Nintendo 64 Mapping Workshop

[Peardian]

Now that I've finished Paper Mario, I figure it's time for me to share with everyone how I did it. In this workshop, I'll be teaching techniques for ripping and preparing models, as well as answering questions. I will also keep a list of N64 games that people are interested in and whether or not it is possible to map them. And remember, this is for the community, so I want you all to get involved, not just sit around and watch. The more questions you ask and games you bring up, the better I can help you all learn. Now, to kick things off...

0. Programs Required
To rip a map on the N64, you'll need three things: an emulator, a rom, and a 3D modeling program. Obviously, I'm not going to show you where to get these, just which ones you'll need.

With the method I use, you'll actually be using two emulators. The main emulator you'll use to rip the models is 1964, in combination with the Nemu64 Combine Debugger plugin (LemD3D8.dll or later version, make sure you have both .dll files). While this graphics plugin isn't the prettiest thing to view games with, it gives you consistant and reliable VRML rips, which you'll be working with. The second emulator is Project64, v1.6 or later, with the TR64 OpenGL plugin. This will act as your backup for ripping textures, as it can export some textures that 1964 rips incorrectly, as well as ones that aren't used as textures (such as HUD elements). Supposedly, Project64 is able to rip VRML as well, but I can never get the rips it exports to work. You will be working both of these emulators together, ripping models in one and filling in the gaps with the other.

The 3D modeling program you'll be using is Cinema 4D, a very powerful modeling/animation program by Maxon. The version I use is Release 10.5, but anything 10 or later is fine. I know there is a Release 11 out there, which is supposedly much faster at rendering, but I don't know what other changes were made, so it's up to you which version you want to use. If you've used a 3D program before, great. If not, don't worry. Cinema 4D has a great and easy-to-use interface that will have you putting together maps in a couple of days. Feel free to play around in the program if you like, as you want to be as familiar with the controls as you can.

Game List
A Bug's Life - **?
Aero Fighters Assault - X
Armorines - *
Army Men Sarges Heroes - ***?
Army Men Sarges Heroes 2 - **
Army Men Air Combat - X?
Aidyn Chronicles - ** good amount of UV damage to characters (Tropicon)
Banjo-Kazooie - **** heavy vertex shading use, room slides around but static angle/scale, complex textures
Banjo-Tooie - X, doesn't emulate well
Batman Beyond - ***?
Battle Tanx - ***?
Battle Tanx Global Assault - ****?
Bio F.R.E.A.K.S. - X
Blastcorps - **? (mikemc)
Bomberman 64 - ***** impossible to see but theoretically possible
Bomberman 64 2 - *** hard to see with culling issues
Bomberman 64 Arcade Edition - X
Bomberman Hero - X
Buck Bumble - X
Castlevania 64 - ??
Castlevania: Legacy of Darkness - ?? Needs looking into, presumed same as above
Chameleon Twist - X
Chameleon Twist 2 - X
Charlie's Blast Territory - *
Chopper Attack - ***?
Conker's Bad Fur Day - ** no problems it seems
Daikatana - X?
Diddy Kong Racing - **** very quirky, objects are misplaced
Donkey Kong 64 - X, can't emulate right now
Doom 64 - **?
Dual Heroes - ****?
Duke Nukem 64 - X
Duke Nukem Zero Hour - X?
Earthworm Jim 64 - ***?
Extreme G - X, way too much UV damage
Extreme G2 - X, same
Forsaken 64 - *** very small rip area, would take a while
Fushigi No Dungeon 2 - **
Flying Dragon - X
F-Zero X - ***** Heavy UV damage, need to be really close to part of track for it to appear right
Ganbare Goemon - Mononoke Sugoroku - X
Gex 64 - X
Gex 3 - X
Goemon's Great Adventure - X
Glover - X
GoldenEye 007 - X
Harvest Moon - X
Hey You, Pikachu! - X
Hybrid Heaven - **
Jet Force Gemini - **** characters/objects buggy, but terrain is fine
Killer Instinct Gold - X
Kirby 64: The Crystal Shards - ***** Very buggy in anything but Nemu, experts only
Knife Edge - X
Kuiki Uhabi Suigou - X
Legend of Zelda: Majora's Mask - **** complex texture effects (Peardian)
Legend of Zelda: Ocarina of Time - **** complex texture effects, damaged textures, jpegs (Peardian)
Lode Runner 3D - X
Mace the Dark Age - ***?
Mario Golf - X, too much UV damage
Mario Kart 64 - ** complete (Peardian)
Mario Party - X
Mario Party 2 - X
Mario Party 3 - X
Mega Man 64 - X
Mischief Makers - ?? maybe possible?
Mystical Ninja Starring Goemon - ** some quirky textures that will require a bit of editing, but nothing too bad
Neon Genesis Evangelion - X
Nightmare Creatures - X
Ogre Battle 64 - * no problems, complete (Tropicon)
Onegai Monsters - X
Paper Mario - *** some UV issues, vertex shading use, complete (Peardian)
Perfect Dark - **?
Pokemon Snap - ***, possible only through cheats
Quake - X
Quake 2 - X
Quest 64 - *** hard to see with culling issues
Rakuga Kids - *?
Rampage - X
Rampage 2 - X
Rat Attack - X
Rayman 2 - X
Resident Evil 2 - X
Re-Volt - X
Road Rash 64 - X
Robotron 64 - X
Rocket - ****? UV damage
Rush 2 - *** same as first
San Francisco Rush - *** map slides around with camera but remains in proper orientation
San Francisco Rush 2049 - *** same as first
Shadow Man - **?
Space Station Silicon Valley - ***** heavy UV damage to terrain, many objects aren't models, trouble emulating, insane mappers only
Star Fox 64 - X
Super Mario 64 - X
Super Smash Bros. - ** some UV damage, complete (Peardian)
Tonic Trouble - ***? possible issues with emulation
Turok: Worlds - **?
Turok 2 - **** heavy UV damage
Turok 3 - ***? dumps a lot
War Gods - **?
Wonder Project J2 - X
Worms - X
Xena - **?
Yoshi's Story - X
Zool - **?

Table of Contents
0. Programs Required
1. Ripping VRML Data with 1964
-Q: other programs?
-setting up the VRML folder
-working Lemmy's plugin
-resource organization
-ripping process
2. Problems in Ripping VRML
-unmappable games
-damaged UV data
-vertex shading
3. Ripping Images with Project 64
-types of images in the N64
-working TR64
-transferring save files from 1964 to PJ64
4. Getting Started in Cinema 4D
-basic C4D layout
-rescaling models with Coordinates panel
-correcting the lighting
-saving
5. Basic Scene Correction
-catching missing scene elements
-basics of Materials
-Color Channel
-Alpha Channel
-handling unneeded objects & materials
6. Axis Control and Optimization
-object axis
-Center Axis To
-Optimize
7. Tiling Textures
-Seamless tiling
-material tag attributes
-controlling Offset, Length & Tiles
-dealing with x-only/y-only seamless tiling
-dealing with edge smearing
8. Model Modification and Scene Setup
-model editing modes
-Camera objects
-Perspective and Parallel view
-picking a viewing angle
-handling backface culling
9. Rendering To File
-Render view
-Render settings
-Reference Spheres
-organizing your renders
10. Shadows and Overlapping Objects
-overlapping horizontal planes
-overlapping vertical planes
11. Cheat Codes
-using cheat codes
-codes for different game versions
Q: Only some models show up?
12. UV Editing
-BP UV Edit Layout
-UV Mesh basics
-UV Commands
-Transform
-repairing UV damage
13. Advanced Texture Techniques
-compound textures
-Fusion shader
-Projector shader
-gradients
-setting up a 3D linear gradient
-gradient + fusion
-mask gradients
-Texture Axis Mode
14. Dealing with Backfaces
-coloring backsides black
-Create Polygon tool
-Connect function
-covering areas outside the walls
-Extrude tool
-objects sticking through walls
-Visibility tag
15. Artistic Renders
-Light types
-Floor object
-Shadow types
-Phong shading
-Global Illumination
-Sky object
-Ambient Occlusion
-Compositing tag
16. Shadow Catcher
-Installing plugins
-Shadow Catcher
17. Connect Object
-phong across multiple objects
-Connect Object
-Quick Shading
18. Reflective Objects
-fake reflections on the N64
-Environment Channel
-Reflection Channel
Tip: Infinite Lighting, Infinite Angle
19. Vertex Weight Maps and Vertex Shading
-Vertex Maps
-Colorizer shader
-Layer shader
-Coloring with vertex maps
Tip: Snapping Settings

That's the end of the planned lessons. However, I will continue to maintain this thread as long as necessary, answering questions and helping with maps.

[Trop]

Great, where was I.  My biggest problem currently is just getting C4D to read the VRML rips.  The message I get is "Error reading VRML2".  I'm looking for a different copy if C4D now in case that's the problem.  And of course the games I'm going for here are Ogre Battle 64 and Aidyn Chronicles.

[TerraEsperZ]

I'm certainly interested on a technical level. However I doubt I'd actually want to spend the required time to map a level. If that doesn't bother you, then please impart us with your knowledge oh enlightened one

[Revned]

Peardian, have you ever tried using Blender? It's free, so it might be a better choice for people who don't want to go to lengths to "acquire" Cinema 4D. It's been years since I've used it, though, so I don't know how well it will work for this.

[Peardian]

@Tropicon: It might be an error in the technique you're using to rip the VRML. That is what today's lesson is about.
@TerraEsperZ: It's certainly not for everyone, but it's always good to learn.
@Revned: I have, and it is the most confusing and unfriendly modeling program I've ever used. People are welcome to use other modeling programs, but they'll be on their own when it comes to correcting the data.


1. Ripping VRML Data with 1964
The first step in putting together a 3D map is to rip the models you'll be working with. But before you even start up the emulator, you must create the folder where the resources will be dumped. Create a folder immediately in your hard drive directory, and name it VRML. (Example: C:\VRML) This is where 1964 will put all the scenes and textures it rips. Make sure you keep this folder empty most of the time, because otherwise you will get batches of files mixed up. Now, onto the actual ripping.

After you've made sure that the Nemu64/Lemmy's Video Debugger plugin is selected and you've started the game, you are ready to begin. With the shortcut ctrl+V, you can easily bring up the video's control panel, which you will be using a whole lot. This is what it looks like...

[img width= height=]http://img13.imageshack.us/img13/245/lemmy.png[/img]

There are quite a few options, but only a few will be used. Checking the Export VRML box is what will make the emulator begin dumping models and textures. You will be turning this on and off repeatedly, so learn that shortcut! The only other option of interest is CIMG Tracing. Sometimes, there will be some kind of texture effect used that will completely blank out the screen, making it impossible to see. Checking this option lets you see again, though you probably won't be able to see whatever effect was causing it. The only time I've encountered this problem so far is in Paper Mario, with the fire effects used in Dry Dry Ruins and Bowser's Castle. It is likely you will encounter something similar in other games. There's nothing wrong with leaving CIMG Tracing checked the whole time.

However, you should not leave Export VRML checked the whole time for a very good reason. When it is checked, it exports the whole scene, models and textures, to the VRML folder every frame. Because of this, the game will run incredibly slow, depending on how many models and textures there are. Additionally, because 1964 overwrites files each frame, you will only have one model scene at a time. For this reason, it is best to keep organized folders in which to store your scene rips and the textures that accompany them. I keep a folder for each separate room in the game, grouped together by the level/area and again by the name of the game. Obviously, the folders should be kept outside of the VRML folder, otherwise you might accidentally grab them by mistake when handling rips. But enough about organization, let's get to the key moment.

The actual rip takes only a second, and a minimal amount of effort. Look for a location that gives you as much of a view of part of the room as you can. Games will often not render models that are out of the screen view, and as such they will not show up in the rip. This means that for big/complicated rooms, you will have to take multiple scene rips to get everything in the room. The amount of offscreen models varies from game to game, ranging from very little (Mario Kart 64) to entire rooms (Kirby 64). Anyway, find a good spot to rip and 1. pause the emulator. Open up the control panel and 2. check Export VRML. Now, after you've closed the control panel, 3. quickly unpause and pause the emulator. This will cause the emulator to advance only a single frame, exporting all the model and texture data to your folder. However, you can let it advance multiple frames if you want to rip a specific pose of a moving object or rip multiple frames of an animated texture. Different frames are stored as separate images, so they will not overwrite each other. Be sure to 4. uncheck Export VRML before you resume playing. Now let's see what you've ripped!

When you check the VRML folder, you'll see a collection of bitmap files and one file named output.wrl, which is the scene file. All the bitmaps will be named with hexadecimal addresses and come in pairs, one for color and one for alpha. Every bitmap will have an alpha channel, though most of them will be completely solid (black). It is important that you do not change the names of the files, as the output file will use those when C4D assigns textures to objects in the scene. All the images you see in the folder are only the ones used by the models in some way. Keep in mind that not every image you see in the game is a model texture. Thinks like the HUD and background images will be absent from the scene and thus the folder. You'll be using Project64 to rip those. Congratulations, you've just ripped a VRML scene!

2. Problems in Ripping VRML
Because Nintendo 64 emulation is far from perfect, there are many problems that can wrong in ripping, some of which are not the fault of the emulator. The biggest problem is that the game internally renders models in a certain way. Instead of a camera moving around a static room, the camera is static and the whole room moves around. Some games take this even further by rescaling the models to simulate zooming. This makes the game unmappable without the use of some kind of tool to access the original model data. Games with this problem will be noted in the game list with an X, for unmappable. Chances are most of these games will still have minimap images that can still be ripped, but the 3D models will be out of grasp for now.

The other problems are to be blamed on the software and are more controllable. Textures on models are controlled by UV data, which tells how the textures are tiled and stretched for each face of the polygon. Sometimes, this data is misinterpreted by the emulator, leading to faulty rips. This usually manifests itself as the texture being skewed and tiled 20 times in some direction, making it stand out easily. Other times, the error will only become apparent upon opening the output file, like in Paper Mario where all character models have their UV map scrunched up into a tiny little square in the bottom left corner of the texture. Either way, these errors are correctable, often easily. This merely adds a degree of difficulty to preparing the models. It is mostly a problem with character models and rarely affects terrain.

The third problem is merely a deficiency in Cinema 4D and the VRML format. Nintendo 64 makes use of something called vertex shading, in which shadows, lighting, and sometimes color is put on a model with gradients. Color values are given to each vertex of a model, and a gradient between them is created on the polygon face. This is used to simulate lighting, as well as color grayscale images and create shadows. The VRML format only retains this data for objects that don't have textures assigned to them, but Cinema 4D cannot handle the data at all. It will merely assign a single color to the whole object, taken from the first vertex of the object. There is a workaround for this, but that's for another lesson. This is possibly the most time-consuming issue to deal with.

Update: I've found a way to do vertex shading, but it isn't easy. See the lesson on it for more details.

[Peardian]

Since there were no questions, it's time to carry on. The next lesson is...

3. Ripping Images with Project 64
Ripping a scene with 1964 will get you the textures you need to render the scene, but often this is not enough for a map. Background images like in Paper Mario and HUD elements are often not part of the scene and need to be ripped separately. Additionally, some textures will rip incorrectly with 1964 because of how they're stored. See, the Nintendo 64 has four types of images, with three of them being the most common. There are the standard color bitmaps, jpeg images (only seen it in Ocarina of Time so far), and two types of grayscale bitmaps. The first type of grayscale bitmaps are easily recognizable in that both the color and alpha channels will be identical. The second kind have different color and alpha channels, and it is this kind that 1964 has trouble ripping. The alpha data will be misinterpreted as red levels in the otherwise-grayscale color image, leaving the alpha image completely empty (white). If you find an image like this in the scene dump, you'll know it's time to get Project 64. Jpeg images have to be extracted with a special tool made by _demo_, which I have yet to find.

The process of ripping textures with Project 64 is very simple. Just open up the Graphics Plugin options after you've started the game (sorry, no shortcut). There are only four checkboxes and a dropdown list. If the game isn't playing and it tells you to change the uCode, this is where you do it. I don't really know how to determine which uCode you need for which game, so it's a bit of trial and error. Anyway, the option you'll mainly be interested in is Enable Texture Dumping. This automatically dumps each and every image loaded in memory into the tiles folder inside the main Project 64 folder, which you must create yourself. Unlike with dumping VRML, this will not affect performance or take a lot of power. However, it will only load each unique image once, and as such you will only have one chance to dump the desired image. Make sure you have the dumping option checked before you reach the room containing the desired image. Keep in mind that the earlier you check the box, the more images you'll have to weed through, but the safer you'll be in capturing it. Missing your chance to rip the image means you'll have to close the emulator and start all over again. Depending on what the image is, you might want to even start the dump as soon as you load the game.

When you check the tiles folder for the images you've ripped, you'll notice that Project 64 handles things differently. Images are simply labeled Texture#### or Tile#### and come in groups of three as opposed to pairs. Not only that, but all the images are flipped vertically and the alpha channels are inverted as compared to the dumps made by 1964. The middle file in each group is an Info file, which gives data about how the image i used in the game. You can delete all of these, as they are of no use to us. An easy way to delete them is to use the Search function and search the folder for names with the word "info" and delete all the results. What is left are all bitmaps, in two categories. Images labeled Tile#### are the raw image files, and the ones labeled Texture#### are how they're used in the scene, after some distortion. The images you'll normally be looking for are the Tile bitmaps, but that's not to say the Textures aren't useful. Sometimes they reveal how a particular image is tiled or how it is combined with other bitmaps (more on this another time). Be sure to look over these before you toss them, as they might point out something you failed to notice. Since not all images are used as textures, there will always be less Texture images (if not the same). The count on the files resets each time the emulator is closed, so be sure to keep the folder clean after each use. Oh, and since there is a size limit to images in the N64, larger images like backgrounds will always come in pieces, either as chunks or strips. Prepare to do a bit of piecing together. And that's all for ripping with Project 64. Not much to it. Just remember to vertically flip the images you get from it before using them in the scene.

But what's that, you say? Project 64 and 1964 are using separate save files and it impedes your progrss? Yes, this is true, but there is a way around this. Upon inspecting the save folders of both emulators, you'll notice that 1964 adds hexadecimal gibberish to the end of savefile names while Project 64 does not. Because of this, it's easy to copy the save over to Project 64, but not the other way around. Simply remove the numbers at the end after copying it to Project 64's folder. I don't know how well the process works in reverse, but I imagine you'd have to use the number found on 1964's original save file to append. This trick will save you a great deal of effort, especially when it comes to lengthy RPGs like Paper Mario. With the last of the image resources ripped,

[Trop]

A few things.  When exporting VRML from 1964 how do you select the folder to export to?  And my project 64 folder has no "tiles" folder.

[Piranhaplant]

I am having a similar problem, I get the output.wrl file, but no textures.

[Peardian]

@Trop: As far as I know, you cannot configure the export folder. It will always dump everything to C:\VRML, or whatever your drive is. As for Project 64, yes, I forgot to mention you will have to create the tiles folder as well.
@Piranhaplant: That is strange. I'm assuming it's not the game, because I'm sure you wouldn't try ripping a 3D scene from a 2D game like Yoshi's Story. Which game is it? And does the output file work? (Try opening it in C4D.) How big is the file?


I'll postpone the lesson for a little so I can try and solve these problems.

[Trop]

I know where the VRML dumps go so it's fine.

OK I created a tiles folder and as soon as I started doing something it did indeed fill up with tiles.  Not ones I can use though.  I'm walking around on a field in Ogre Battle 64 and all it dumps are fonts and what I guess you would consider sprites.  I thought I needed the textures I'd be covering the 3D models with.  I'm even using the uCode for Ogre Battle.

Still working on getting a new C4D.

[Peardian]

@Trop: Like I said, the textures will only be dumped by Project 64 the first time they are loaded into memory. If you're walking around in a field, they've already been loaded and it's too late to dump them. And, remember, Project 64 is for supplemental image ripping. You only have to turn to it if you need extra images or encounter an erroneous texture.

What version of C4D did you have before?

[Trop]

*sigh* I was using 10.5.  I just picked up 11.5 and it gives me the same error.  And I'm even using a different computer!  All my ripped VRML's work in that stupid Cosmo Player for Firefox.

[Peardian]

@Trop: Then it might be a problem with your process and not C4D. Try reading over the lessons again to make sure you have the right versions of everything and are following the proper steps. Even though I bolden and underline things, be sure to read everything in between. Anyway, I pushing on with the next lesson. I need to take advantage of the time I have left at home.

4. Getting Started in Cinema 4D
Now that you've ripped the scene's VRML data and the textures that go with them, you're ready to start work on the models. I recommend you keep up the emulator (paused) while working in C4D. Even with 512MB of RAM, you can still support both programs fine. I also recommend you associate the .wrl filetype with C4D, if you haven't already. Double clicking the file is a lot more convenient then having to drag the file into C4D or use the Open dialog. The layout for C4D is pretty simple. In the left-center, you have the main viewing window. To the right is the list of objects, on the bottom is the list of materials in the scene, and in the corner is the attributes about whatever tool or object you've clicked last. Between the materials and info are fields related to the selected object.

The first thing you'll see when opening the scene file is most likely going to be graphical gibberish or empty space. This is because the VRML scene is ripped at 1/1000 the size you want to work at and rendering objects tends to break down at this size. The fix for this is quite simple, and dealing with it will become routine for you soon enough. To start, 1. choose the NemuModel object in the object list. This is what's known as a Null Object, an invisible container to group objects together. All scenes ripped by Lemmy's plugin will be contained within a null object, along with a camera and light object. Next, look down at the Coordinates box. This box normally displays things like the current spatial coordinates, size, and axis rotation. There is one field for X, Y, and Z in each group. You can change two of the groups to display other information, and that's what you're going to do.

[img width= height=]http://img258.imageshack.us/img258/5989/scales.png[/img]

In the center box, 2. choose Scale in the Coordinates box if you haven't already. The size numbers will change to display three 1s. The numbers will always show 1, but you can use this field to rescale models. With the null object still selected, 3. put 1000 into the three Scale fields and hit Apply. This will resize the model to a workable scale. Keep in mind that hitting enter will apply the changes you put in the fields, so use tab to switch from field to field. You will notice that the gibberish will clear up, but the view has not changed. You'll also notice that somethings will appear dim. This is because of the default light. It is included in every scene rip at the origin, which often is in the middle of the floor or a little above it. This makes it hard to see and thus hard to work with. In order to fix this, 4. choose the light object to bring up its attributes.

[img width= height=]http://img94.imageshack.us/img94/3449/lightwp.png[/img]

The Ambient Illumination option makes the singular point light light up everything evenly, which is just what you want. So, 5. check the Ambient Illumination option to light up your scene. If you have a version of C4D earlier than 10.5, you might also have to choose the Constant Shading option in the main view window's Display menu. This gives you an evenly lit view like with Ambient Illumination, but it does not affect the actual render. The shortcut for this N~E, which means you press the N key, followed by the E key. This shortcut is useful for when you want to check a scene without modifying the light. Now that you've made these fundamental changes to the scene file, I recommend you save it. Make sure you pick a meaningful name, keeping in mind that you will be creating individual scenes for each separate room in the game. Something like "room #" is okay if you keep the folders organized, but an even better name would be something like "desert #" or "bowser #", including part of the location's name. Cinema 4D's proprietary format is .C4D, which only it can read. With this modified scene file created, you can get rid of the original output.wrl file.

Now that the scene file is set up and saved, let's take a look at it. Any easy way to do this is to select the null object and press the S key, a shortcut which repositions the camera to fit the selected object (or group of objects) in the center of the view. Looks good? Maybe, maybe not. Depending on the game, the scene will look more or less like how you see it in-game, usually less. You might notice a lot of grayscale objects and a lack of alpha transparency for textures. We'll handle this in the next lesson. In the meantime, I recommend you become familiar with C4D's camera controls. They are located in the upper righthand corner of the main view window, and you will be using them a whole lot.

[Trop]

Yeah, I'll try more angles.  I don't want to hold up your posts.

[Piranhaplant]

@Piranhaplant: That is strange. I'm assuming it's not the game, because I'm sure you wouldn't try ripping a 3D scene from a 2D game like Yoshi's Story. Which game is it? And does the output file work? (Try opening it in C4D.) How big is the file?

The game I am trying to rip from is Mario Kart 64 (not for mapping purposes), and the output file doesn't work and is 390kb.  I also forgot to mention that output.wrl is not being put into the C:\VRML folder, it is being dumped directly onto the C drive.