• View
• History
• Print

Main / P2

Project 2 - Motion Capture Display and Editing

For this project, you'll build a program read in and display motion capture data, and do some basic motion editing.

There are two reasons for this project:

1. You will learn a lot about working with motion capture data (including how to deal with poorly documented file formats, issues in dealing with hierarchies and rotations, ...)
2. You will build infrastructure that will be used in the 3rd project, which will require you to do something even more interesting with motion capture data.

1.  The Basic Requirements

The basic things your program must do:

1. Read in BVH files (a standard skeletal animation data file format).
2. Display the motions in an interactive viewer
3. Support some simple editing operations
4. Have at least some advanced features
5. Be able to write motions back out

For an interactive viewer, your program must (at least):

1. Display a 3D view of the character
2. Display some other information about the motion
3. Have a groundplane
4. Support real-time playback (frame rate locked to the appropriate speed)
5. Support scrubbing (a slider to move scan through times)
6. Camera control (to look around)

More details follow...

2.  Some ground rules

You will work in assigned groups of 3. It is up to you how to divide the labor. You could even decide to have everyone write their own program and just turn in the one that turns out the best (this is not a recommended strategy).

1. Your program must run on the machines in the Starsky and Hutch lab (1358 CS). Exceptions to this rule can be arranged ahead of time (contact me before April 1st), but only if there's a good reason (like you want to use a platform that can't be installed in 1358).
2. You may use any language you want (C++, Java, Matlab, Python, C#, ...).
3. You may use any windowing package (FlTk, Swing, SWT, XNA, ...) you like.
4. You may use any graphics library you like (OpenGL, JoGL, Java3D). If the library contains support for skeletal animation, you should make sure that you implement some of the key components (like the BVH parser) yourself. You are welcome to use the 559 sample projects as a starting point.
5. You may use any math/data structure libraries you like.
6. If you want to use an animation library, talk to me.
7. We will do demos in class on April 17. Your "demo grade" is based on what you show at this time.
8. You are responsible for using source code control, and backing up your development. If you need AFS space for a CVS/SVN repository let me know at the beginning of the project.

Be warned: I cannot provide you with any help for tools other than C++/FlTk. If you choose another development environment, you're on your own. Also, it is your responsibility to install any tools you need so you can run your program on the 1358 computers. If you get hung up because of issues in the tools you've chosen, expect little sympathy.

Another warning: there is a chance that we will mix up the groups for the 3rd project, so other people may be looking at your code.

3.  Reading Files

Your program must read "Biovision hierarchy files."

Motion capture file formats are documented in a set of notes by Jeff Thingvold (now at Motion Analysis) (here). These notes are over 10 years old, but they are still useful - people still refer to them. The BVH file format documentation is somewhat vague, but its a vague format. The "original, official" specification was some hand sketched notes by some people at BioVision (I have a copy of those notes, and they aren't too helpful).

In the directory p:/graphics/public/Data/Motion, you will find lots of BVH files to experiment with. You will see all sorts of different skeletal configurations, joint types, and other variations of the file formats.

In the ideal world, you would be able to read any BVH file we threw at your program. More realistically, we would like your program to understand some subset of them. The larger the subset, the better (and this will be rewarded in grades). Some ground ground rules:

• Your program should not crash on a file that it cannot understand.
• Your program must read at least 10 of the files in the Motion directory. You can pick which 10. This is a minimum - in the ideal case, you should be able to read any of them. In a realistic case, for any motion in the directory, your program should read and display it correctly with high probability.
• You must, in your documentation, describe the limits of the reader. It is much better to say "my reader only supports ZXY euler angles for joints" than to have a reader that mysteriously doesn't work some times.

Some reasonable simplifying assumptions you might make include: supporting only certain types of euler angles, or only supporting a fixed topology. (the latter is a severe limitation, and I do not recommend it).

There are sample files designed to stress test a parser in:

    /p/course/cs777-gleicher/public/p1-test

Your program should be able to read in multiple motions. Its nice if you can display them at the same time (switchable).

Bonus Features:

• Read in ASF/AMC Format. This is particularly nice since there is a huge database of motions for it. On the other hand. Plus there is plenty of example code out there for it.

4.  Displaying the Motion Data

You must provide an interactive viewer for displaying the motions that you read in. Your viewer should use OpenGL to display things, and provide the ability to play the motion at "frame rate" as well as the ability to "scrub" (interactively move through frames). You must provide some interactive camera controls so that the user can control the view.

The nicer that you draw things, the better. Drawing lines between the joints is the easiest, but drawing some "bones" (like ellipsoids) looks a lot better. Skinning looks the best, but that's a really advanced feature.

Drawing a groundplane and hack (drop) shadows are an easy way to make things look a lot better.

You program must be able to place the camera in a position where it can see the whole motion (this requires you to figure out the spatial extents of the motion). When a new motion is read in, the camera should be positioned so that the whole motion is visible.

The user must be able to interactively manipulate the camera.

You must put in a timer to control framerate. Your program must offer the option of displaying animation at 30fps. You might want to allow other options (like just blasting the animation as fast as possible, or allowing for slow motion viewing).

Bonus Feature: A tracking camera that follows the character as it moves. This take a little thought to do well - it can't be too bouncy, or spin too fast, ...

5.  Displaying Other Information About the Motion

You should do some things to help make the motion easier to see. The user should be able to switch these on and off.

• Drawing "traces" that sweep out the paths of the end-effectors. (this is good to make sure you can compute the forward kinematics).

Bonus Feature: Try being creative in given tools to help the user visualize the motion. Consider how to draw "strobes" (multiple frames at once).

6.  Basic Editing Features 1: Concatenation

These are the building blocks of more advanced techniques. By themselves they aren't too interesting, but they will be useful later.

Your program must be able to concatenate sub-clips of the motions loaded in together. For example, you should be able to do something like "add frames 30-60 from motion A to the end of motion B". For now, the specification of what motions to concatenate can be manual - and it doesn't have to have a good user interface. And you can assume that the characters have the same topology/morphology (best if you provide an error if they don't). If the limb lengths aren't the same, you should just use the skeleton from the motion that you're appending on to. You can consider retargeting later.

You must do at least the most basic version of this, better if you can do the harder versions:

1. just make a longer motion
2. rigidly align the second clip so that its root position/orientation matches the end of the first
3. do some form of blending/filtering to get rid of discontinuities / make better transitions

You'll probably want 2 and 3 for project 3, so its better to do it sooner rather than later.

Bonus Features: you could support various experiments with number 2 and 3 to see how the various methods compare. For example, rather than using the exact alignment in number 2, you could adjust the angle a little bit (how much cand you do this before its noticable?). Or, are blend transitions better than cut/filter transitions?

7.  Basic Editing Features 2: Blending

Basic blending - where you take a "linear" (linearly in quotes, since you might want to average rotations in a better way - like using slerp for quaternions) combination of each pose - was used a little bit in the previous section, but your program should be able to do it more generally. You should also be able to have the blend weights change (like go from 0 to 100% of motion B in order to make a transition).

You should be able to do something like "blend of frames 10-40 of motion A with frames 50-80 of motion B with weights 30/70 and put the result at the end of motion C". Just like with concatenation, there are issues of positional alignment, ...

An easy addition: stretch the length of one motion so that it has the same length as the other. (timewarping is a bigger bonus thing). One thing you can do with this: blend a pose with a motion, so you can blend a sitting down pose with walking motion to get, well, something funny.

For now, you just need to support simple blending. Looking ahead, you might want to handle blending where the motions are aligned better.

A simple alignment (recommended): align the first frames of the two motions to be blended (for example, aligning both to the end of the motion that the result is being concatenated onto). You might also consider incrementally blending the global position orientation. (but that's a more advanced feature).

Bonus Features: Fancier blending (implementing more of registration curves), or coming up with better transition mechanisms.

8.  Other bonus features

While "big things" are better left for project 3, here are some that aren't too unreasonable:

Path Editing: You can provide the user with a way to edit the path of the motion (e.g. if the character was walking in a straight line, you can bend it and have it walk along a curve).

Path Analysis: A lot of social scientists are interested in what are the natural paths people take when they talk around. If you just look at the root trajectory, you see the oscillations side-to-side for each stride, so its hard to really assess the curvature. You can try various things for this.

Footplant identification: you can determine when the feet are supposed to be planted on the floor. Its easy to get it right 95% of the time. Tricky to get it 99%. I don't think its possible to be right 100% of the time. Footplant solving (fixing errors in the footplants) is probably too much for this project.

You can talk to me about any of these, or even other ideas.

9.  The Deadlines

On April 17th you'll turn everything in:

• You'll give a demo to show off what your program does.
• You'll provide documentation (preferably as a page on the wiki)
• You'll turn in a copy of your code.

Each group must have a wiki blog. You should post to it regularly. However, you must at a minimum make the following postings. (be sure to use the word "MILESTONE" in the posting title)

• MILESTONE 0: On Monday (can be anytime, before Tuesday morning) 3/30, you must make a posting as to what language/platform you will be using
• MILESTONE 1: On Friday 4/3, you must post a screen shot that you actually have a 3D world, and also indicate your progress towards reading a file.
• MILESTONE 2: On Friday 4/10, you must post a screenshot of your system displaying a motion (there should be a character in it). This posting should also confirm that your system is properly displaying animations, and give some sense of what motions it works on/doesn't work on. You should also indicate what kinds of bonus features you're trying to get done for the following week.

Note: if you aren't meeting the milestones (e.g. deciding what language platform by 3/30, or showing motion by 4/10), you will have a really hard time getting the project done.

10.  Handing It In

The handin for project 2 will consist of 3 parts:

1. The demos (given during class time on Friday, April 17th) where you'll show off your program. Expect to have it read in and display a bunch of motions, to be asked to show off all the basic features (interactive display, ...), and to give examples of the simple editing and blending operations. If you have bonus features, you'll have to tell me about them and show them off as well.
2. Handing in all code - turn in everything needed to build and run your program, including a README file saying what the pieces are (and what instructions for building it). Make a ZIP file and put it somewhere public on AFS.
3. Make a class wiki page (it can be in one person's page group, or in the "Contrib" group) documenting your program (as described below).

1 is scheduled. Officially, 2 and 3 are due at the same time. However, I will accept late things (within reason - up until Tuesday, April 21st) without penalty. One person in your group should send me email when the wiki page and the zip file are ready (telling me where they are).

In your wiki page writeup, please include:

1. A description of the limitations of your parser.
2. A list of the basic requirements that you have working. For the basic editing operations (blending and concatenation), describe the variant the you've implemented (how you do alignment, what kinds of transitions, ...)
3. A list of all of the bonus features that you've implemented.
4. A list of any outside code that you used (libraries, ...), and any resources you consulted (web pages, etc) for figuring the project out. (I'm curious what's useful)
5. A description of how to use your program. Describe the basic interface, and how to access the features. In particular, be sure to describe how you specify what blends/concatenations to do.
6. Screenshots. In particular, if you can show off examples of where blending/concatenation works, that would be particularly good. (its hard to show in a static image).

Either as part of your writeup, or as a seperate wiki page, you also need to provide an assessment of every other group's project (you'll see them during the demo session). The big question for you: does it look like this project will be a good platform for doing Project 3? (implementing a more advanced motion editing/synthesis technique)

Note: we'll be moving on to project 3 soon, so you might want to get this done quickly.