• audio

• video

• still pictures

• animated computer graphics

• on a computer

Each of these has several possible formats and there are many details concerning those formats. Multimedia production involves

• creating sound, picture, video, or graphics files
• editing, combining, archiving and finding, and manipulating such files
• incorporating them in Web pages
• incorporating them in computer programs

It is this last part that concerns us in a Windows course. There are many programs on the market to assist with the first three (e.g., Adobe's entire product line).

When you see a movie, the list of credits is often hundreds of pages long. Software in the future may have equally long lists of contributors, many of whom will be involved in the creation or editing of media files. Developments along these lines have been slower than predicted seven years ago.

Because of the wide variety of file formats and software to play them, Microsoft has developed the Windows Multimedia Library to provide a uniform API (Application Programmer's Interface) for multimedia programming. The purpose of this lecture is to explain how to use the Multimedia Library.

Common audio formats:

.wav (plays only under Windows)

.mp3

.midi (MIDI = musical instrument digital interface)

.au

audio CD format

Windows can play a .wav file directly, using the PlaySound function. This is really using the Multimedia Library, but the programmer may not even know this. No direct calls to Multimedia Library functions are made. To play files in the other formats, you will need to use the Multimedia Library functions directly.

You can create a .wav file using WaveEdit, which comes with

There is also analog audio. At present, telephones carry analog audio. Digital audio, in which the signal is encoded in IP packets which can be transmitted over the Internet and reassembled for playing, is coming: they say the telephone network will switch over to IP packets within five to ten years, at which time there will be no real distinction between the Internet and the telephone network. At present, this technology is already in use for Internet telephony and Internet radio, but the sound is sometimes interrupted or delayed, as packets are reassembled.

.bmp (Windows bitmap, usable only under Windows)

.jpg or .jpeg (should be used for photographic-style images on the Internet;

but should not be edited as it loses information with each edit)

.gif (should be used for images on the Internet with only a few colors)

There are also vector graphics used e.g. by Adobe Illustrator, etc. There is no good way to incorporate an Illustrator file in a Windows program directly--it has to exported in .bmp format.

To display a picture in a Windows program, it must be a .bmp file. Therefore, the programmer must be able to convert images from other formats to .bmp format. Internet Explorer can save a .gif or .jpg as a .bmp file, so it is easy to convert images found on the Internet.

To create a .bmp file as original art, one can use a paint program, such as Windows Paint (which comes with Windows in the Accessories folder). More commonly, people use Adobe Illustrator and when the file is finished, they export it in .bmp format.

We have already studied how to display .bmp files in a Windows program--you don't need the Multimedia Library to do it.

• Analog
• Digital

Analog digital is what currently is shown on television. The signal is either in NTSC format (in America) or PAL format (in Europe).

NTSC = National Television Standards Committee. The picture is encoded as frequency modulations in an electromagnetic wave.

Digital video encodes the picture as a sequence of 0's and 1's, and then encodes this sequence in an electromagnetic signal.

Digital video can be stored in a .avi (avi = audio-visual interface) file and played under Windows. The problem with this is the huge size of .avi files. A ten-second video clip can easily occupy a megabyte.

To play analog video under Windows, you need special hardware often known as a "video board". (Not to be confused with the "video card" or "graphics card" that drives the monitor, and is present in every computer.) A video board permits you to plug an ordinary video cable into the board and watch TV in a window on the computer (and, if you like, record it into a computer file.) These boards have gotten quite a bit cheaper in the last few years and some (but not all) Mac computers come equipped with them. You can buy a PC, equipped with two large hard disks and a video board and all the right software, for under $5000 now.

You can also plug a camcorder into the board and record directly onto your hard disk. (Again, it takes a lot of space.) People who do this often have several large hard disks in the same computer, some of which are used only for storing video files. Software that comes with the board may permit the conversion of its native file format to .avi files.

Computers now can be equipped with DVD drives which assist in playing digital video. The format of such video signals is neither that of .avi files nor NTSC video.

We have been waiting now more than six years for the promised convergence of TV and computers. This must await either the presence of video boards on every computer, or the digitization of television. Both are slower coming than was predicted.

This lecture comes from the two books, Microsoft Windows Multimedia Programmer's Workbook and Microsoft Windows Multimedia Programmer's Reference, as well as from Visual Studio Help. All these sources are "old" by now, but the Multimedia Library hasn't changed. Only the ways of accessing it have changed. You still need to understand the concepts.

This lecture only gives the basic architecture of the multimedia API and one example, enough to enable you to play a video clip or midi file in your Windows program.

Media files take up large amounts of disk space. Here are three examples, each of which takes one megabyte:

• 10 or 15 seconds of digital video

• 40 seconds of 8-bit waveform stereo audio

• three 640 by 480 bitmaps in 256 colors

MIDI and mp3 files are much smaller, but one does not see digital compression schemes for video with corresponding efficiency.

Since comparatively few images can easily fill up hundreds of megabytes, image-intensive software often leaves the images on the CD or DVD, and then requires that disk to be in the computer. The capacity of a CD-ROM is about an hour of video or some fifty thousand still images. A DVD disk can hold twice as much--enough for a movie.

There is a special function in Win32 which plays .wav files. You do not need the full Windows Media Control Interface.

You can either play the sound by mentioning its filename (in which case the file needs to be distributed with the program) or by making the file into a resource with a line like

MyBells WAVE c:\sounds\bells.wav

in the resource file.

PlaySound

The PlaySound function plays a sound specified by the given filename, resource, or system event. (A system event may be associated with a sound in the registry or in the WIN.INI file.)

BOOL PlaySound(
  LPCSTR pszSound,  
  HMODULE hmod,     
  DWORD fdwSound    
);
 

Parameters

pszSound

A string that specifies the sound to play. If this parameter is NULL, any currently playing waveform sound is stopped. To stop a non-waveform sound, specify SND_PURGE in the fdwSound parameter.

Three flags in fdwSound (SND_ALIAS, SND_FILENAME, and SND_RESOURCE) determine whether the name is interpreted as an alias for a system event, a filename, or a resource identifier. If none of these flags are specified, PlaySound searches the registry or the WIN.INI file for an association with the specified sound name. If an association is found, the sound event is played. If no association is found in the registry, the name is interpreted as a filename.

hmod

Handle of the executable file that contains the resource to be loaded. This parameter must be NULL unless SND_RESOURCE is specified in fdwSound.

fdwSound

Flags for playing the sound. The following values are defined:

SND_ASYNC

The sound is played asynchronously and PlaySound returns immediately after beginning the sound. To terminate an asynchronously played waveform sound, call PlaySound with pszSound set to NULL.

SND_FILENAME

The pszSound parameter is a filename.

SND_LOOP

The sound plays repeatedly until PlaySound is called again with the pszSound parameter set to NULL. You must also specify the SND_ASYNC flag to indicate an asynchronous sound event.

SND_MEMORY

A sound event's file is loaded in RAM. The parameter specified by pszSound must point to an image of a sound in memory.

SND_NODEFAULT

No default sound event is used. If the sound cannot be found, PlaySound returns silently without playing the default sound.

SND_NOSTOP

The specified sound event will yield to another sound event that is already playing. If a sound cannot be played because the resource needed to generate that sound is busy playing another sound, the function immediately returns FALSE without playing the requested sound.

If this flag is not specified, PlaySound attempts to stop the currently playing sound so that the device can be used to play the new sound.

SND_NOWAIT

If the driver is busy, return immediately without playing the sound.

SND_PURGE

Sounds are to be stopped for the calling task. If pszSound is not NULL, all instances of the specified sound are stopped. If pszSound is NULL, all sounds that are playing on behalf of the calling task are stopped.

You must also specify the instance handle to stop SND_RESOURCE events.

SND_RESOURCE

The pszSound parameter is a resource identifier; hmod must identify the instance that contains the resource.

SND_SYNC

Synchronous playback of a sound event. PlaySound returns after the sound event completes.

Return Values

Returns TRUE if successful or FALSE otherwise.

Remarks

The sound specified by pszSound must fit into available physical memory and be playable by an installed waveform-audio device driver. PlaySound searches the following directories for sound files: the current directory; the Windows directory; the Windows system directory; directories listed in the PATH environment variable; and the list of directories mapped in a network. For more information about the directory search order, see the documentation for the OpenFile function.

If it cannot find the specified sound, PlaySound uses the default system event sound entry instead. If the function can find neither the system default entry nor the default sound, it makes no sound and returns FALSE.

Formerly, if you wanted to play a videodisc player from your program, you had to write a videodisc driver that would send commands out the serial port. You used the player's manual to determine what the commands should be.

With the Windows media control interface, you write just one line of code to initialize the player, and one line to make it play or stop playing, or to change frames, etc. The driver is supplied by the device manufacturer. This is similar to the way printers are handled in Windows.

The key to the MCI is

DWORD mciSendCommand(wDeviceID, // an integer ID number

You get the device ID in the first place by sending an MCI_OPEN message to open the device. For this message, wDeviceID can be NULL. You also leave wDeviceID NULL when using a software device that does not need to be "opened", such as the "devices" that play .avi and .wav files.

The parameter wMessage will be passed as an identifier which is defined in the header file mmsystem.h. You will have to include this file whenever you call the MCI functions. Some of the possible values of wMessage are

MCI_OPEN Initializes an MCI device

MCI_PLAY Starts transmitting output data

MCI_RECORD Starts recording input data

MCI_SAVE Saves data to a disk file

MCI_SEEK Seeks backwards or forwards

MCI_STOP Stops playing or recording

MCI_PAUSE Pauses playing or recording

MCI_RESUME Resumes playing or recording

MCI_CLOSE Closes an MCI device

You may read about the "command-string" interface to MCI. There is a function mciSendString which can be used to send strings encoding MCI messages. These strings are parsed by MCI and converted to messages. This interface is primarily for Visual Basic users, or non-programmers using multimedia "authoring systems". As a C programmer, you should ignore it.

To use the MCI functions directly, you have to link to the multimedia library explicitly. It is called winmm.lib. Choose Project | Settings | Link and type in winmm.lib.

Today (2002) it is perhaps easier to use the multimedia library by means of an ActiveX control provided by Microsoft that encapsulates the functionality of the library. Instead of making direct calls to MCISendCommand, you call the methods of the control, for example, SetCommand("Play");

Some documentation about any ActiveX control can be found by pressing the "More Info" button when the control is selected in the Gallery (while adding it to your project). But in the version of MSDN (Visual Studio Help) that I have (and that is in the lab) there is no documentation of the Multimedia ActiveX control. In any event, if you want to play media files under program control, it is only a matter of simplifying the syntax slightly-the basic steps involved in programming multimedia have not changed. If, on the other hand, you want to give the user control over the medium (e.g., play and pause) then the ActiveX control is somewhat easier as it saves you the work of designing the controls for play, pause, etc.

I will show one programming example with MCISendCommand, and then one example with the Multimedia ActiveX control.

First we will go over the code to play an avi file. Then we will construct a simple application to demonstrate it.

// avi.cpp: Code to play an avi file using the MCI

// M. Beeson, 1995

#include <stdafx.h>

#include <mmsystem.h>

#include "mciavi.h" /* a Microsoft file */

#include "digitalv.h" /* also a Microsoft file */

#include "avi.h"

#define ERRORSIZE 200

static void report_error(DWORD err);

static int videoID;

void CloseAviDevice(void)

/* Called from window procedure on

WM_MCINOTIFY message. Close the Avi device.

*/

{ mciSendCommand(videoID,MCI_CLOSE,0,NULL);

}

int PlayAviFile(HWND hwnd, LPSTR avifile, RECT *r)

/* Play the specified file in window hwnd, in the rectangle r

specified in client coordinates. If r==NULL then the

default location, taken from the .avi file itself, is used.

Return 0 for success, nonzero for failure.

*/

{ MCI_DGV_OPEN_PARMS mciopen;

MCI_DGV_PLAY_PARMS mciplay;

MCI_ANIM_WINDOW_PARMS mciWindow;

MCI_ANIM_RECT_PARMS mciRect;

DWORD err;

mciopen.lpstrDeviceType="avivideo";

mciopen.lpstrElementName=avifile;

mciopen.hWndParent=hwnd;

/* open mci 'device' */

err = mciSendCommand(NULL, /* device id should be NULL for MCI_OPEN */

MCI_OPEN,

MCI_OPEN_TYPE|MCI_OPEN_ELEMENT|

MCI_ANIM_OPEN_WS,(DWORD)(LPVOID)&mciopen

);

if(err)

{ report_error(err);

return 1;

}

videoID=mciopen.wDeviceID;

/* videoID will be the first parameter of

further mciSendCommand calls */

/* The next task is to tell it where to put the video image.

The MCI_WINDOW message is used to supply an application-created

window to the device and to change the display characteristics

of an application-supplied or default display window */

mciWindow.hWnd = hwnd;

mciWindow.lpstrText = NULL; /* put a caption here if you wish */

err = mciSendCommand(videoID,

MCI_WINDOW,

MCI_ANIM_WINDOW_HWND | MCI_WAIT,

/* The first of these flags tells it that

mciWindow.hWnd is a valid window handle, of the

window in which we want the image displayed.

Use the MCI_ANIM_WINDOW_TEXT flag if you

are supplying a caption in the lpstrText field.

*/

(DWORD)(LPVOID)&mciWindow

);

if(err)

{ report_error(err);

mciSendCommand(videoID,MCI_CLOSE,0,NULL);

return 1;

}

if(r==NULL) /* rectangle not specified, inquire from the file */

{ err = mciSendCommand(videoID,

MCI_WHERE,

MCI_ANIM_WHERE_SOURCE | MCI_ANIM_RECT,

(DWORD)(LPVOID)&mciRect

);

if(err)

{ report_error(err);

mciSendCommand(videoID,MCI_CLOSE,0,NULL);

return 1;

}

/* after the MCI_WHERE message, mciRect.rc describes the

clipping rectangle of the video device */

}

else

mciRect.rc = *r;

err = mciSendCommand(videoID,

MCI_PUT, /* used to specify the destination rectangle */

MCI_ANIM_PUT_DESTINATION | MCI_ANIM_RECT,

/* these flags tell it to use mciRect.rc for the

destination rectangle. */

(DWORD) (LPVOID)&mciRect

);

if(err)

{ report_error(err);

mciSendCommand(videoID,MCI_CLOSE,0,NULL);

return 1;

}

/* Now play the .avi file and notify the parent window after completion */

mciplay.dwCallback=(DWORD) hwnd;

/* This specifies the window to be notified when the

file completes playing. The MCI_NOTIFY flag in the

next line tells it to send this window an MM_MCINOTIFY

message when the playing is finished.

*/

err = mciSendCommand(videoID,

MCI_PLAY, /* at last */

MCI_NOTIFY,

(DWORD)(LPVOID)&mciplay

);

if(err)

{ report_error(err);

mciSendCommand(videoID,MCI_CLOSE,0,NULL);

return 1;

}

/* Do NOT close the device; this function will

terminate while the file is still playing */

return 0; /* success */

}

/*__________________________________________________*/

static void report_error(DWORD err)

/* Look up the error string associated with the error

whose number is err, and present it to the user

*/

{ char errorbuf[ERRORSIZE];

mciGetErrorString(err,(LPSTR)errorbuf,ERRORSIZE);

MessageBox(NULL,errorbuf,"MCI DEVICE ERROR",MB_OK);

}

Now, to create a simple application to demonstrate this code.

1. Create a new SDI project avidemo. Use Project | Add to Project | Files to add avi.cpp and avi.h (which contains prototypes of PlayAviFile and CloseAviDevice.
2. Add the library winmm.lib under Project | Settings | Link. Check that the project builds.
3. Map the WM_FILEOPEN message in the view class. Call the common dialog to open a file (as we studied before). The code is given below.
4. In the same message handler, after the file is open, call PlayAviFile to play it.
5. Now, you can play the file--but you can't play it twice, because we didn't close the device. You will get an error message the second time. We must close the device upon receipt of the MM_MCINOTIFY message in the view window. But, of course Class Wizard doesn't list this message, so we have to map it by hand. In the MESSAGE_MAP section of avidemo.cpp, add the following line:

ON_MESSAGE(MM_MCINOTIFY, CView::OnMM_MCINotify)

Note, the nearby lines say ON_COMMAND, but you want ON_MESSAGE.

6. Add the declaration of this function in the view's header file:

protected:

//{{AFX_MSG(CAvidemoView)

afx_msg void OnFileOpen();

//}}AFX_MSG

void OnMM_MCINotify();

DECLARE_MESSAGE_MAP()

Only one line was added, but nearby lines are shown to show the place.

7. Add the implementation of the message handler, which is just one line of code, CloseAviDevice().

8. Test the application. Open a .avi file; when it finished playing, see if you can successfully play it again.

Here comes a surprise: The very same code that plays an AVI file will also play a MIDI file. The multimedia commands are quite general. You only have to change one line, namely the name of the device to be opened. Instead of "avivideo" you need the name of a MIDI player. The same should be true for mp3 files.

The Multimedia MCI control manages the recording and playback of multimedia files on Media Control Interface (MCI) devices. Conceptually, this control is a set of push buttons that issues MCI commands to devices such as audio boards, MIDI sequencers, CD-ROM drives, audio CD players, videodisc players, and videotape recorders and players. The MCI control also supports the playback of Video for Windows (*.avi) files.

When you add the Multimedia MCI control to a form at design time, the control appears on the form as follows:

The buttons are defined as Prev, Next, Play, Pause, Back, Step, Stop, Record, and Eject, respectively.

Your application should already have the MCI device open and the appropriate buttons in the Multimedia MCI control enabled before the user is allowed to choose a button from the Multimedia MCI control. In Visual Basic, place the MCI Open command in the Form_Load event.

When you intend to record audio with the Multimedia MCI control, open a new file. This action ensures that the data file containing the recorded sound will be in a format compatible with your system's recording capabilities. Also, issue the MCI Save command before closing the MCI device to store the recorded data in the file.

The Multimedia MCI control is programmable in several ways:

• The control can be visible or invisible at run time.
  
• You can augment or completely redefine the functionality of the buttons in the control.
  
• You can control multiple devices in a form.

If you want to use the buttons in the Multimedia MCI control, set the Visible and Enabled properties to True. If you do not want to use the buttons in the control, but want to use the Multimedia MCI control for its multimedia functionality, set the Visible and Enabled properties to False. An application can control MCI devices with or without user interaction.

The events (button definitions) of the Multimedia MCI control are programmable. You can augment or completely redefine the functionality of these buttons by developing code for the button events.

The MCI extensions support multiple instances of the Multimedia MCI control in a single form to provide concurrent control of several MCI devices. You use one control per device.

Distribution Note When you create and distribute applications that use the Multimedia MCI control, you should install and register the appropriate files in the customer's Microsoft Windows System or System32 directory. The Package and Deployment Wizard included with Visual Basic provides tools to help you write setup programs that install your applications correctly.

Here is a sample project: give the user a control with which to play an audio CD (along the lines of WinAmp, etc.)

1. Create a new SDI project, based on CFormView. Be sure that you ask for ActiveX support.

2. Project | Add to Project | Registered Active X Controls.

Select the Microsoft Multimedia Player control and added it to your project.

3. An icon for the new control appears on the toolbar in the dialog editor. Drag one to your form.

4. Use Class Wizard to add a control variable m_mmcontrol corresponding to the new control.

5. In OnInitialUpdate add the following:

6. Map the WM_DESTROY message and make the handler look like this:

Note, you have to put the new commands above the call to the parent class OnDestroy, not below where AppWizard suggests.

7. Now you can test the program: Put a CD in your computer's CD player. Wait till the default player starts to play it, and then shut it off. Now run your program. It should play your CD and stop playing it when you close your program. Without the OnDestroy handler, it goes on playing after your program closes.

8. Use Class Wizard to map the messages corresponding to the Play and Pause buttons, and add the following handlers:

Now the user can pause and restart the music.