Software Engineering
    Computer Science
    Design Patterns
    Philosophies
    Standards
    Licenses
    Trends
    Integration Patterns

Multimedia

Updated: 2022-03-26

Video Format

  • Raw
    • REDCODE RAW (R3D)
    • ARRIRAW
  • Camera rendered file formats
    • QuickTime (MOV): container format, from apple, used in Nikkon and Canon. can use different codecs, e.c. H.264
    • MPEG-4 (MP4/M4V)
    • Material eXchange Format (MXF): sony
    • Audio Video Interleave (AVI): similar to MOV, out of favor

Compression Standards

  • H.264/AVC (MPEG-4 Part 10)
    • Advanced Video Coding
    • supports resolutions up to 4096×2304, including 4K UHD
  • H.265/HEVC (MPEG-H Part 2)
    • High Efficiency Video Coding
    • supports resolutions up to 8192×4320, including 8K UHD
  • H.266/VVC (MPEG-I Part 3)
    • The final standard was approved on 6 July 2020
  • ProRes
    • developed by Apple

Intraframe vs Interframe

  • Intraframe: every frame is encoded individually
  • Interframe: only the differences between frames are encoded

What is Transcoding?

Transcoding: covert from one video format to another.

Adobe Media Encoder is a good tool for transcoding.

Resolution

UHD=ultra HD

  • Full HD: 1920×1080, 1080p
  • 4K UHD: 3840×2160, 2160p
  • 8K UHD: 7680×4320, 4320p

Netflix

Netflix has been using H.264/AVC almost exclusively. However, users who download Netflix shows to most Android devices instead receive content encoded with VP9 an open source video codec developed by Google that uses a bunch of advanced encoding tricks to deliver the same video quality with significantly less data, or a better video qualities with the same amount of data.

H.264: BASELINE vs MAIN vs HIGH

TL;DR

  • High: Best for big screens with a good decoder. Use HIGH profile for final storage of your edited material
  • Main: Good for more modern smartphones and tablets. Almost deprecated.
  • Baseline: Compatible with most smartphones and tablets, including older mobiles; live recording

BASELINE

Pros:

  • Easy to decode & encode video streams (compared to the rest of the H.264 profiles) - biggest selling point
  • Better in every way than MPEG2 in terms of packed quality

Cons:

  • Made sense a few years back when you needed a PC to play custom H.264 HD content
  • The processing power of modern CPUs can easily handle more challenging settings - enter MAIN and HIGH profile

MAIN

Pros:

  • B-frames and CABAC entropy encoding make enable this profile on average 15% better than BASELINE in terms of packed quality
  • Originally targeted at professional motion picture distribution so they made sure it is well up to the task.

Cons:

  • Slower to encode than BASELINE due to increase complexity of the encoding tools
  • Not suitable for live recording
  • Essentially made obsolete by the advent of the HIGH profile

HIGH

Pros:

  • Addition of 8x8 intra prediction modes over what MAIN profile offered, made this the "go-to" profile when it comes to professional motion picture distribution. This encoding tool really helps keep things pristine in scene cuts places or fast moving action.
  • You want this one!

Cons:

  • Slowest of the three when it comes to encoding times. However, not much harder to decode then MAIN. If your setup handles MAIN profile flawlessly, then in theory it should sustain a HIGH profile stream of the same visual quality (meaning that for the same quality the HIGH profile stream would have a lower bit-rate therefore allowing the CPU to focus more on math processing)
  • Definitely not suitable for live recording. (Although there is a form of HIGH profile consisting only of INTRA frames that is used by a certain camcorder manufacturer to record live footage.)

Cloud Storage

  • Amazon Drive(Prime Photo): unlimited photo storage for Prime members, no compression, RAW count as photo

Video Editing

Free:

Commercial:

  • Avid Media Composer
  • Adobe Premiere
  • Final Cut Pro

Photo Editing

Free Lightroom alternatives:

raw pedia: http://rawpedia.rawtherapee.com/Main_Page

Gears

  • Monopod
  • Tripod: iFootage Cobra
  • Stabilizer: Zhiyun/DJI

Photo Format

  • HEIC vs HEIF vs HEVC vs JPEG
  • Apple ProRAW combines the information of a standard RAW format along with iPhone image processing, which gives you more flexibility when editing the exposure, color, and white balance in your photo. ProRAW files are 10 to 12 times larger than HEIF or JPEG files.

To choose between HEIC and JPEG on iOS: Settings -> Camera -> Format

To choose between HEIC and JPEG when transfering out: Settings -> Photos -> Transfer to Mac or PC

XMP files: the data standard for Adobe’s Extensible Metadata Platform.

IPTC: the photo metadata standard. https://www.iptc.org/standards/photo-metadata/

Apple Photo has a "Export IPTC as XMP" option when exporting photos. If selected, a XMP file will be exported along with the photo file.

HDR

HDR: see something very dark and something very bright in the same image

Relative to

  • low dynamic range (LDR)
  • standard dynamic range (SDR)

two ways to achieve HDR:

  • computer renderings and images resulting from merging multiple low-dynamic-range (LDR) or standard-dynamic-range (SDR) photographs.
  • using special image sensors, such as an oversampled binary image sensor.

Stock Video and Music

IMAX

而 35 毫米胶片的摄影机提供的水平分辨率约为 6000,一个 IMAX 摄影机提供的水平分辨率为 18000。数字 IMAX 分辨率为 2K

Terms