4 The asdcplib library is a set of objects that offer
5 simplified access to files conforming to the sound and
6 picture track file formats developed by the SMPTE Working
9 This work was originally funded by Digital Cinema
10 Initiatives, LLC (DCI). Subsequent efforts have been funded
11 by Deluxe Laboratories, Doremi Labs, CineCert LLC, Avica
12 Technology and others.
14 **The asdcplib project was originally housed on SourceForge.
15 The project has moved to http://www.cinecert.com/asdcplib/
17 The project formerly depended upon the mxflib project. Because
18 of its focus on covering the whole of the MXF specifications,
19 mxflib is considerably larger and more complex that what I
20 require for this application. For this reason I have created
21 a dedicated MXF implementation that is now part of this
22 library. Special thanks to Matt Beard and Oliver Morgan for
23 their great work and support.
25 Thanks also to the members of the SMPTE DC28.20 packaging
26 ad-hoc group and the members of the MXF Interop Initiative
27 for their encouragement and support. Special thanks to
28 Jim Whittlesey and Howard Lukk at DCI for proposing and
29 supporting this project.
34 This library is intended (but of course not limited) for
35 use by developers of commercial D-Cinema products. It is
36 designed to be easily integrated into a wide variety of
37 development environments. Commercial users are strongly
38 urged to use static linking (at least where you use this
39 library) to prevent malicious in-field replacement of
40 critical system modules. This recommendation should be
41 considered wherever Open Source or Free software is being
42 used in conjunction with critical security parameters, such
43 as cryptographic keys.
45 The author strives mightily to provide an API that is completely
46 independent of operating system and other library dependencies,
47 and which allows selective replacement of some modules for
48 local needs. Specifically, the essence parsers and OpenSSL
49 crypto functions can be replaced by linking to alternative
50 implementations of the ASDCP:: objects which provide those
53 AS_DCP.h contains the entire API. You do not need to read
54 any of the other files, except maybe asdcp-test.cpp which
55 contains detailed usage examples of each of the API's
56 services. The KM_* files may be of interest for general
57 development support, but may be ignored if all you want
58 is simple AS-DCP support.
63 GNU make is required to build asdcplib. The makefile will work
64 on win32 systems that have Cygwin and MSVC. Other win32 gmake
65 packages may or may not work depending upon the availability
66 of standard POSIX shell commands. You will need gcc to rebuild
69 OpenSSL is also required. See http://www.openssl.org/
71 If you are building on win32 or a unix with no OpenSSL library
72 support, you will have to also obtain and build OpenSSL. Unpack
73 it into the same parent directory as asdcplib/, and rename (or make
74 a symlink to) the directory as 'openssl':
78 drwxr-xr-x 1 jhurst None 0 Feb 3 16:37 asdcplib
79 drwxr-xr-x 56 jhurst None 0 Feb 2 16:35 openssl
81 Expat is optionally required for writing Timed Text Track Files. See
82 http://expat.sourceforge.net/
84 To build, type 'make'. There are several test targets but you
85 need to assemble a set of test files to use them.
87 I have tested this build on win32, Linux, OpenBSD and Darwin
88 platforms. Others may work as well.
95 asdcp-test - Writes, reads and verifies AS-DCP (MXF) track files.
97 kmfilegen - Writes and verifies large files using a platform-
98 independent format. Use it to test issues related to large files.
100 kmuuidgen, kmrandgen - generate UUID values and random numbers.
102 wavesplit - Splits a WAVE file into two or more output files. Used
103 to untangle incorrectly-paired DCDM sound files.
105 blackwave - Write a WAVE file full of zeros, Used to make filler
106 tracks (though you would be better off modifying asdcp-test if
107 this is a common use case).
112 Currently, the API documentation is mostly in AS_DCP.h. Read
113 that file for a detailed description of the library's capabilities.
114 Read asdcp-test.cpp for library usage examples. The command-line
115 utilities all respond to -h.
119 2007.10.29 - Bug fixes v.1.2.17
120 o Changed Result_t implementation to use int instead of long, which
121 was causing trouble on some 64 bit platforms.
122 o Fixed EKLV HMAC. Broke backward compatibility with older Interop
123 files. To validate these files use asdcplib-1.1.14. This should
124 not cause too much trouble since files with broken and non-broken
125 HMAC have been in the wild for years without trouble.
126 o Fixed HMAC sequence numbering in encrypted stereoscopic files.
127 o Finished stereoscopic test targets in the makefile.
128 o Fixed the win32 build, now expects VS2005 compiler by default,
129 use WITH_VC6=1 top get VS6 flags.
132 2007.10.22 - Timed Text, Stereoscopic Picture and Bug fixes v.1.2.16
133 o Significant API changes have been made. Please read all entries
134 in this changelog to be sure you understand the changes. Also
135 note that some changes have been made to LS_MXF_SMPTE files that
136 are incompatible with earlier releases (e.g., EKLV HMAC). If
137 you are looking for a stable interop release, use v.1.1.14.
138 o Fixed RFC 2104 HMAC implementation for LS_MXF_SMPTE only. The
139 broken implementation has been maintained for Interop mode.
140 o Added support for draft SMPTE 429-5 Timed Text Track File. This
141 is still waiting for official SMPTE ULs, so do not use it for
142 shipping products. An XML parser is needed to create a Timed
143 Text Track File; Expat is now an optional part of the build.
144 Make with WITH_XML_PARSER=1 to link with Expat. If you do not
145 link with expat, you will get an error when using the TimedText::
146 DCSubtitleParser class. See also S429-5-cgi.cpp for an example
147 that shows how to serve plaintext MXF file elements directly via
149 o Added support for draft SMPTE 429-10 Stereoscopic Picture Track
150 File, including the JPEG Interop version. This is still waiting
151 for official SMPTE ULs, so do not use it with LS_MXF_SMPTE for
153 o Refactored the following files as a side-effect of the above
154 work: AS_DCP_JP2K.cpp AS_DCP_MPEG2.cpp AS_DCP_PCM.cpp
155 AS_DCP_MXF.cpp AS_DCP_internal.h MXF.[h|cpp] MXFTypes.[h|cpp]
156 Metadata.[h|cpp] h__Reader.cpp h__Writer.cpp klvwalk.cpp.
157 WARNING: While significant effort has been extended to make sure
158 that these changes do not affect existing stable file support,
159 users are cautioned to test this release thouroughly.
160 o Added a large set of filesystem path manglers to KM_fileio.h. See
161 path-test.cpp for example usage. The path manglers have not yet
162 been tested on win32 platforms (they are currently used only by
163 the Timed Text module.
164 o The PathIsFile(), PathIsDirectory() and FileSize() subroutines
165 have been modified to accept const std::string& instead of
167 o Added namespace and parsing support (Expat) to Kumu::XMLElement
168 (currently used only for Timed Text support). Also added some
170 o Altered MXF::UTF16String to use mbtowc() and wctomb().
173 2007.03.31 - Bug fixes v.1.1.14
174 o Fixed KeyFrameOffset value in MPEG wrapping to have negative
175 value. This is probably not yet complete for handling all
176 types of GOPs. Please send chunks of MPEG-2 VES that you
177 find which break this. Thanks to Doremi.
178 ** no other file format changes in this release **
179 o Fixed error in RIP interpretation when reading arbitrary (i.e.,
181 o Fixed a memory leak in ASDCP::MXF::OPAtomHeader when used
182 in read mode. Thanks to Mahesh Bajaj for pointing out this
183 bug and the one above.
184 o Removed asserts from KM_fileio, replaced with RESULT_WRITEFAIL
185 return value statements.
186 o Added -s and -p to the makefile install target.
187 o Altered ByteString behavior to use target Length() in copy
188 operations (instead of Capacity()).
189 o Added new Set() method to ByteString.
190 o Fixed a bug in ByteString::Unarchive() that caused the operation
191 to fail when the buffer was smaller than the read (i.e., when
192 Capacity() was called).
193 o Added IdentifierList class to KM_util.h.
194 o Changed some Error() messages to Debug() in Wav.cpp
195 o Revived jp2k-test.cpp and asdcp-mem-test.cpp (they both had
199 2007.02.15 - Bug fixes v1.1.13
200 o Removed 'VDescObj->SampleRate.Numerator = VDesc.FrameRate;'
201 from MPEG2_VDesc_to_MD() in AS_DCP_MPEG2.cpp, was line 76.
202 o Added KM_TEST_NULL_STR_L() and KM_TEST_NULL_L() macros to
203 KM_log.h. These versions log the location of the null value.
204 Macros are now used in any module that includes KM_log.h.
205 o MPEG2 VES with run of zero values at the head is now OK.
206 o Increased VESHeaderBufSize to 16K.
207 o Added makefile support for local OpenSSL in ../openssl, if
209 o The Kumu::PathIs*() functions now return false if a null or
210 empty string is given (used to be an assert).
211 o Cleaned up the install target in the makefile.
212 o Fixed SMPTE 429-6 HMAC -- FIPS 186-2 implementation was
213 laughably incorrect. Thanks to Doremi for pointing this out.
214 o Removed default parameter to HMACContext::InitKey().
215 o Cleaned up messages and CLI arg handling in asdcp-test.
218 2006.11.19 - Mo better stuff v1.1.12
219 o Changed read-only Result_t accessor methods to const.
220 o Added Base64 (-B) option to kmrandgen.
221 o Removed 16-bit alignment restriction from kmrandgen.
222 o Improved WAV file extraction speed (Thanks to Jim Radford
223 for pointing this out).
224 o Added single-channel split for WAV extraction (asdcp-test -1).
225 o Fixed remainder bug in h__RNG::fill_rand().
228 2006.11.03 - Bug fixes v1.1.11
229 o Increased index table entry list size to 5000.
230 o Added length checking to TLV writer (returns error if TLV
231 payload exceeds 64kB).
232 o Fixed partition header and RIP errors related to 2-partition
233 files (MXF Interop mode).
234 o Added -t option to asdcp-test (SHA-1 digest with Base64 output
236 o Fixed Sub Descriptor reference bug (Thanks to Denis Leconte
237 for dogged determination).
238 o Added directory-of-wav detection to RawEssenceType()
239 o Modified MXF::Partition::AddChildObject() to only generate
240 a UUID if the InstanceID is unset.
241 o Added ComponentMaxRef & ComponentMinRef to RGBAEssenceDescriptor.
243 o Added detection of 2K/4K jp2c, writing correct 4K metadata.
246 2006.10.05 - Bug fixes v1.1.10
247 o Changed RM_RELEASE to RL_RELEASE in MXFTypes.h.
248 o Changed the MXF writer to use RL_RELEASE (was RL_DEVELOPMENT).
249 o Really fixed source reference chain.
250 o Updated JP2K file package label.
251 o Changed location of JPEG2000PictureSubDescriptor in the
252 header (was erroneously before Preface).
253 o Altered LS_MXF_INTEROP to produce 2-partition files.
256 2006.09.25 - Bug fixes v1.1.9
257 o Fixed SourcePackageID value. All files will be 'original',
258 i.e. SourcePackageID will be all zeros. Let me know if you
259 want to set SourcePackageID.
260 o Fixed compiler warnings on some Linux platforms
261 o Fixed the build so that BUILD_DIR is no longer created
263 o Added duration detection to the raw essence parsers. The
264 MPEG parser uses a nasty approximation so don't use it
265 without paying close attention to the result.
266 o Modified PCMParserList to make it more useful as a base
268 o Fixed bugs and re-organized command-line help in asdcp-test
270 o Fixed two-partition file reads.
271 o Fixed Win32 PRNG initialization.
272 o Renamed asdcp-lf-test as kmfilegen.
273 o Added kmrandgen and kmuuidgen.
274 o Added string retrieval mechanism to Result_t.
275 o Refactored Kumu::Identifier and its sub-classes.
276 o Altered Kumu::PathIsFile to return true when the path
277 is a symbolic link (unix only).
278 o Altered Kumu::FileWriter::OpenWrite to use file creation
279 mode 0664 (was 0644) (unix only).
280 o Added Kumu::WriteStringIntoFile() subroutine.
283 2006.04.05 - Bug fixes and new stuff v1.1.7
284 o Fixed a bug in the MPEG parser that caused it to fail when
285 handling start codes spanning buffer boundaries
286 o Added wavesplit and blackwave utility programs
287 o Added support for revised SMPTE HMAC key derivation when
289 o Refactored platform compatibility and general utilities
290 into a new sub-library "Kumu". There are no new build
291 steps or dependencies, but some important things have
293 + Result_t is no longer an enum, it is now a class.
294 Library result codes are now declared as const objects
297 const Kumu::Result_t RESULT_FORMAT (-101, "The file...");
299 The macros ASDCP_SUCCESS and ASDCP_FAILURE still work
300 the same way thanks to an operator overload for type long.
301 See KM_error.h for more information.
302 + The logging interface has been moved out of AS_DCP.h
304 o Some of the command line utilities that were using headers
305 other than AS_DCP.h have been changed to use the Kumu
306 equivalents. If you have code based on those utilities, you
307 will have to update by hand.
308 o Added new types to the EssenceType_t enum.
309 o The guard macro for Win32 code has changed from WIN32 to
313 2006.03.2x - new stuff
314 o Proper handling of stream-id byte of essence UL values
315 o writes 3-partition files, reads 2-part or 3-part
318 2006.03.16 - bug fixes plus
319 o Removed SMPTE_LABELS compile-time option. The reader will now
320 silently accept either SMPTE or MXF Interop labels, the writer
321 can be instructed which to use at runtime. Default is Interop.
322 o Added an AIFF reader. Support is preliminary, it works with the
323 AIFF files I have on hand.
324 o More code refactoring. More to come.
327 2006.03.09 - full read-write
328 o Removed ASDCP_WITHOUT_OPENSSL compile-time option.
329 o Full read/write now working on new MXF library
332 2005.00.00 - A New Hope
333 o The temporary mxf-lite has been removed. MXF files are now
334 managed via the objects in KLV.h, MXFTypes.h MXF.h and
335 Metadata.h. This release does not support writing MXF files.
336 o Fixed a header interpretation error in the Wav parser.
339 2005.00.00 - The Reformation
340 o Removed mxflib as a dependency by forking the necessary
341 functions and placing them in the mxf-lite subdirectory.
342 Please note that the very heavy modifications done here
343 render all comparisson to mxflib code a substantial task.
344 All errors are now mine and users are warned not to bug
345 Oliver or Matt for help with this code. The version of
346 mxflib at the time of the fork was: 0.5.1.3.
349 2005.06.03 - bug fixes v0.10.18
350 o Updated UL batch to include GC UL.
353 2005.05.27 - bug fixes v0.10.17
354 o Un-did essence container and compression descriptor changes.
355 The default build reflects MXF Interop decisions as of 26 May.
356 o Added note about build versions to README (see above).
357 o Added warnings to SMPTE_LABELS builds.
358 o Fixed JP2K essence container label.
361 2005.05.02 - bug fixes v0.10.16
362 o Reorganized internal files, added file reader object, added OS
363 portability header, removed and renamed some files. If you have a
364 patch against previous versions of the source, you should check
366 o Added RGBA attributes to JP2K descriptor.
367 o Changed interface to CodestreamParser.
368 o Added JP2K parser implementation. It is parsing each frame but is
369 not yet being used to populate the descriptor.
370 o Added 48fps option for `asdcp-test -p`.
371 o Added picture rate constants to AS_DCP.h (23.976, 24, 48).
372 o Added sample rate constant to AS_DCP.h (48k).
373 o Changed asdcp-test to encrypt picture headers by default
374 (plaintext offset will be 0), added -E option to allow
378 2005.04.28 - bug fixes v0.9.15
379 o The XML descriptors for the crypto DMS have moved in mxflib to
380 the file DMS_Crypto.xml (they were in DMS_DCPENC.xml). Older
381 installations should update the file from mxflib.
382 o Added Close() and Seek() to ~MyFileWriter(), cleaned up headers
383 o Added UUID generator output mode (-u).
384 o Added -S option to extract PCM essence into stereo wav files
385 o Added more UL testing and conformance checking.
386 o Added macro SMPTE_LABELS which causes the library to be built
387 with SMPTE (as opposed to MXF Interop) labels. This is not
388 set by default, and currently only affects the PCM container
389 label and encrypted element label.
390 o Cleaned up the GNUmakefile test targets, the source files
391 are now named with the TEST_FILE_PREFIX macro.
392 o enabled 23.976-framed PCM (2002 samples per frame)
393 o The size of the asdcp-test frame buffer for picture essence
394 may now be set from the command line (-b). The default is 4MB.
395 o h__Reader::ReadEKLVPacket() now tests the UL (duh) and switches
396 on the value, allowing plaintext and ciphertext frames to
397 be mixed in the file.
398 o Fixed error in UUID generator format.
399 o JP2K files now use the GenericPictureDescriptor to store
400 ContainerDuration and SampleRate. SampleRate is mapped
401 to EditRate in the PictureDescriptor struct. This fixes
402 the second caveat from the 0.8.13 release.
403 o Fxed bug in PCMParserList that was miscalculating the extent
404 of a PCM sample. This bug did not affect API users, it was
405 only present in asdcp-test.
406 o Fixed EditRate on PCM files (was showing sample rate)
407 o Fixed Encrypted Essence Container UL
408 o Fixed BlockAlign value for PCM essence
410 - The following changes were provided by Jeff Loewenguth
412 o Moved the DMS CryptographicFramework entry from the material
413 package to the source package
414 o Fixed erroneous Source Essence Container Label value
415 o Fixed broken sort of JP2K frames in JP2K parser
416 o Added FindFrameGOPStart() method to the MPEG2 MXFReader
417 o Added missing length values for EKLV packets without HMAC
418 o -x with JP2K essence writes to files with 6 digit names
420 o The Key ID may now be specified as an argument to asdcp-test
424 2004.12.30 - bug fixes + wav files v0.8.14
425 o Added WAV file write to asdcp-test (uses mxflib::waveheader_t).
426 o Three-partition files reading properly. adscplib still writes
428 o Changes in the mxflib WAV essence parser API had broken
429 asdcplib's ability to read essence from a WAV file. I have
430 fixed this bug, but at the expense of breaking compatibility
431 with older versions of mxflib. Beware!
432 o Removed redundant (but working) bin-text-bin conversions.
435 2004.12.23 - JPEG 2000 support v0.8.13
436 o Reads/writes JPEG 2000 essence in plaintext and ciphertext
437 with the following caveats:
438 - The Picture Essence Descriptor is empty.
439 - Because there is no essence descriptor, the reader code
440 in asdcp-test has no idea how many frames are in the file
441 and ends with an out-of-bounds frame error. This error
442 is being suppressed in asdcp-test for the current release.
443 o Still broken when reading three-partition files.
446 2004.10.22 - fixes and UL updates v0.7.11
447 WARNING: COMPATIBILITY BREAKPOINT
448 Files created with this and future versions of this library are
449 not compatible with previous versions of this library. As you
450 might suspect, files created with previous versions of this
451 library are not compatible with this and future versions.
453 o h__Reader will now open a three-partitition file (untested)
454 o Moved DMS from Material partition to File partition
455 o Added length fields to appropriate places in EKLV packet
458 2004.10.22 - fixes and UL updates v0.7.10
459 o fixed frame buffer handling of externally allocated buffer,
460 created unit test (asdcp-mem-test)
461 o added operator==() and operator!=() to Rational type
462 o fixed some type-related compiler warnings
463 o asdcp-test -p now works on unwrap
464 o updated some ULs to match documentation (thanks to Arun
466 o canonicalized line endings
469 2004.07.02 - full plaintext + ciphertext read/write v0.6.9
470 o HMAC, plaintext offest and raw ciphertext read supported
471 o back to proper CBR index
472 o MPEG temporal offset working
475 2004.07.01 - plaintext + ciphertext read/write v0.5.8
476 o encryption of MPEG and PCM essence supported with the
479 - no plaintext offest support
480 - no raw ciphertext read support
481 o moved to reflecting the whole KLV triplet in the CBR
482 index (now incompatible with mxflib, still searching
483 for info about what's "right")
484 o awaitng the following fixes/features:
485 - retrieve Temporal Offset from index in MPEG2::Reader
486 - test for correct ULs when reading frame triplets
487 - fix header metadata items for encrypted files
489 - plaintext offest support
490 - raw ciphertext read support
493 2004.06.14 - plaintext read/write w/key generator v0.4.5
494 o Project now uses OpenSSL (tested with 0.9.7d on win32, Linux, Darwin)
495 Use `make ASDCP_WITHOUT_OPENSSL=1` to make plaintext-only version
496 o Accepts interior I frames when parsing MPEG2 VES
497 o Improved error reporting on format errors
498 o Added support for encryption to asdcp-test
499 o Added RNG for asdcp-test (non-production use only, see notes in FortunaRNG.h)
500 o Implemented CBC encrypt, decrypt module
501 o Added partial TemporalOffset retrieval from MPEG2 parser
502 o Fixed win32 binmode bug
505 2004.05.12 - plaintext read/write v0.3.4
506 o Full read/write of plaintext MPEG-2 VES and WAV files
507 (does not yet support mux from or demux to 2 channel pairs)
508 o Builds with autoconf-based mxflib
509 o Added ASDCP_ prefix to macros
510 o Updated documentation, fixed documentation errors
511 o Simplified API for MXF writer setup
512 o Decoupled essence parsers from MXF writers
513 o added raw ciphertext support to FrameBuffer
514 o Cleaned up Get/Set naming confusion
515 o Added missing const qualifiers
518 2004.04.27 - preview release v0.2.1
519 o hasty release for quick review
520 o this release may not build with mxflib using autoconf.
521 o writes plaintext AS-DCP MPEG2 essence files
522 o tested under win32 and linux. FreeBSD and Darwin do not work
525 2004.02.04 - First release (v0.1.1):
526 o asdcp-test is mostly complete. It should correctly provide
527 access to the read and info capabilities of the library.
528 o The MPEG2 reader is functional but not fully tested.
529 o Some file format integrity tests remain to be coded.
530 o The PCM reader is mostly complete, I am having difficulty
531 getting a suitable test file from mxfwrap.
532 o The makefile creates a static library module. The API
533 will cleanly support a DSO (or DLL). Let me know if DSO
534 support is important to you.
535 o Nothing is thread safe. Thread safety was not a requirement
536 in the initial project definition. Thread safety may be added
537 above the mxflib/klvlib level. Let me know if it is important
539 o The files xmldict.xml and types.xml must be in the current
540 directory when the program is run. The files are supplied
541 with mxflib. Runtime location of the files is not currently
542 a project goal. Comments on how best to handle this situation
543 would be appreciated. See AS_DCP.cpp at init_mxf_types()
544 for a more detailed discussion.