Edit AllPages

Is it possible to use the NSMovieView as a surface texture in an opengl application? Could i use the method createTexture:fromView:internalFormat: of class NSOpenGLContext to do this so far my i cant seem to get a to work properly. I want to to this so i use the quicktime movie as a texture in my 3d scene. Thanks.

I don’t know if that could work especially without the view being visible, but if you look at there is a complete example of what you want. It extracts the current frame from a movie and loads it into a OpenGL texture buffer. HaRald

This example is pretty good, except for handling arrow keys. Rather than getting the time scale and dividing by 5 (what a hack!) the app should ask QT for the next “interesting” time. Something like this:

Condensing this down to one function is left as an exercise. Otherwise, I thought the demo was pretty good.

– Mike Trent

the url posted above ( is now defunct. Anyone know where that content might now be found?


OpenGL and QuickTime, on Mac OS X

Mac OS X

CiphSafe Developer Tools LIOService CFFP NFS client NFS server ssh-agent OpenGL Other SW

Java FreeCiv SWebRedirD Miscellaneous

Spam reports Calvin & Hobbes PGP key Home Contents

See Also Summary The Files Startup Code OpenGL Initialization Drawing the Scene Controlling the Demo Cleanup See Also

Be sure to also have a look at: QT ValuePak2 - shows a number of techniques for using QuickTime in games (including, of course, OpenGL use) Apple’s OpenGL Movie - QuickTime with OpenGL, in Carbon Summary

This is a little demo that shows how to use QuickTime code to generate textures for OpenGL, so you can have movies in 3D programs. A couple of screenshots are available (but note they’re a bit boring since static shots of a moving image obviously don’t capture the movement).

What this code does is texture map the movie onto two surfaces, a simple 3D rectangle (front and back) and a sphere. When running, you can pause/play the movie with the spacebar, start/stop the spinning with the S key, fast-forward/rewind with the right and left arrow keys, and change the volume with the up and down arrow keys. In the upper-left corner is a frame rate display, so you can see how different movie file formats and sizes affect performance.

When first run, it will open a window requesting a movie to use. Find one (anything QuickTime supports, but the extensions allowed in the code are mov, mpg, mpeg, mp4, dv, and avi), and choose it. The program will then open a window and start the movie. You can toggle full screen and windowed mode with Ctrl-Cmd-F.

The Files

All the code, and associated Project Builder files, are in the zipfile (15KB). The base Cocoa+OpenGL code is from the ports I’ve done of NeHe’s OpenGL tutorials. Only the interesting code is shown and discussed here.

The compiled application is also available in a zipfile (19KB), if you don’t feel like compiling it.

Startup Code

When it starts, the first interesting bit of code is in GL_QTController.m, in awakeFromNib:

if( EnterMovies() != noErr ) [ self createFailed:@”Failed to initialize QuickTime” ]; This is how QuickTime is initialized. It’s balanced by a call to ExitMovies() in dealloc.

One note, while we’re looking at awakeFromNib, is that the OpenGL view is created with 32 color bits and 32 depth bits, which doesn’t work on all systems (especially the depth bits). Also, when switching to full screen (in setFullScreen:), the mode used is 1280x1024, which also doesn’t work universally. If the program doesn’t start, try changing the depth bits to 16, or 8, or 4, or 2, or even 1. If full screen doesn’t work, try dropping the resolution to one you know your system supports.

Back to initialization, when the OpenGL view is created, the first thing it does (in GL_QTView.m, method initWithFrame:colorBits:depthBits:fullscreen:) is to get the shared NSOpenPanel to ask for a movie.

openPanel = [ NSOpenPanel openPanel ]; [ openPanel setAllowsMultipleSelection:NO ]; if( [ openPanel runModalForTypes:[ NSArray arrayWithObjects:@”mov”, @”mpg”, @”mpeg”, @”mp4”, @”dv”, @”avi”, nil ] ] == NSCancelButton ) [ NSApp terminate:self ]; We quit if the user hits cancel. Once we have a file, we call loadMovie:, which, if you haven’t figured out from the method name, loads the given movie:

Back in initWithFrame:colorBits:depthBits:fullscreen:, we call setupOffscreenGWorld to create an offscreen GWorld into which QuickTime draws the actual video part of the movie:

OpenGL Initialization

OpenGL initialization begins in initGL which first creates a simple, black texture. It’s used to initialize the movie texture so there are no odd pixels in it later when we update the texture.

glGenTextures( 1, texture ); glBindTexture( GL_TEXTURE_2D, texture[ 0 ] ); texSize[ 0 ].width = [ self nextPowerOf2:movieBox.right ]; texSize[ 0 ].height = [ self nextPowerOf2:movieBox.bottom ]; texBytes[ 0 ] = calloc( texSize[ 0 ].width * texSize[ 0 ].height, 3 ); glTexImage2D( GL_TEXTURE_2D, 0, GL_RGBA, texSize[ 0 ].width, texSize[ 0 ].height, 0, GL_RGB, GL_UNSIGNED_BYTE, texBytes[ 0 ] ); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR ); glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR ); free( texBytes[ 0 ] ); To determine the size of this texture, we need to set it to a power of 2 (required for an OpenGL texture), and make sure it is at least as large as the movie; eg, if the movie’s width is 240 pixels, the texture will be 256 pixels wide.

Next, we calculate the ratio between the movie size and texture size. This ratio will be used as part of the texture coordinates when texture mapping in drawRect later.

texMovieWidthRatio = (GLfloat) movieBox.right / texSize[ 0 ].width; texMovieHeightRatio = (GLfloat) movieBox.bottom / texSize[ 0 ].height; Nothing else interesting in initGL except we create a quadric for drawing the sphere.

Drawing the Scene

The first things done in drawRect are to start the movie if necessary, and to move the position to the beginning of the movie if we’ve reached the end.

if( !GetMovieActive( theMovie ) && !moviePaused ) StartMovie( theMovie ); if( IsMovieDone( theMovie ) ) GoToBeginningOfMovie( theMovie ); After this, we enable texturing, select our movie texture, then update that texture if the movie is currently playing.

glEnable( GL_TEXTURE_2D ); glBindTexture( GL_TEXTURE_2D, texture[ 0 ] ); // Select our texture if( !moviePaused ) [ self updateTexture ]; updateTexture is the interesting part here, it’s where the movie image is taken from QuickTime and moved over to OpenGL:

Back in drawRect, we do quite a bit of typical OpenGL code after updating the texture. Set things up, move around, draw a sphere, draw the 3D rectangle, print the current frame rate, and update the rotation variables if spinning is enabled. The only part of this which is interesting is the selection of texture coordinates, where we use the ratios calculated in initGL:

glTexCoord2f( 0.0, texMovieHeightRatio ); glVertex3f( -SCREENX, -SCREENY, SCREENZ ); glTexCoord2f( texMovieWidthRatio, texMovieHeightRatio ); glVertex3f( SCREENX, -SCREENY, SCREENZ ); glTexCoord2f( texMovieWidthRatio, 0.0 ); glVertex3f( SCREENX, SCREENY, SCREENZ ); glTexCoord2f( 0.0, 0.0 ); glVertex3f( -SCREENX, SCREENY, SCREENZ ); The use of texMovieHeightRatio and texMovieWidthRatio cause OpenGL to only use the movie image itself, instead of using the entire texture. Recall from initGL that the texture dimensions are powers of 2, which usually will be larger than the movie. If we used the entire texture, there’d be empty black areas above and to the right of the movie image instead of the movie taking the entire space.

Controlling the Demo

When the spacebar is pressed, pauseMovie is called, which toggles playback. This is accomplished by calling one of two QuickTime functions:

Finally, the up and down arrow keys modify the volume of the movie (if it has an audio track). They each change the volume by 5 units; like backupMovie and advanceMovie, decreaseVolume and increaseVolume only differ in direction.

When the program quits, the cleanup includes (in GL_QTView’s dealloc):

UnlockPixels( gworldPixMap ); DisposeGWorld( offscreenGWorld ); free( gworldMem ); DisposeMovie( theMovie ); CloseMovieFile( movieRefNum ); Basically, we free up the GWorld, free up the memory which was used by it, and dispose of the movie and the movie file reference.

Also, GL_QTController’s dealloc calls ExitMovies() to close out QuickTime. Copyright � 1999-2003 Bryan L Blackburn. All Rights Reserved. —-

Found it in an archive (, mirrored here.