Implementation of various (HTTP based) media ingest protocols.
Original repo: https://github.com/nagare-media/ingest
# run locally
$ make clean build
$ bin/ingest-dev-* -c config/samples/nagare-media/example.yaml
# using Docker
$ docker run --rm \
-v $PWD/config/samples/nagare-media/example.yaml:/etc/nagare-media/ingest/config.yaml:ro \
-p "8080:8080" \
ghcr.io/nagare-media/ingest:dev
nagare media ingest
does not depend on specific operating systems or architectures (apart from some optimization) and should compile to many of Go's compile targets. It is known to run well on darwin/amd64
and linux/amd64
. Binaries for various systems are provided for each release. By default, the local system architecture is used when compiling the code. You can cross-compile by passing OS
and ARCH
to make
:
# build binaries for local system architecture
$ make build
# build for macOS amd64
$ make build OS=darwin ARCH=amd64
The build can be controlled through various additional options. You can list all available options using make info
:
$ make info OS=linux VERSION=1.0.0
Build
OS linux
ARCH amd64
Version Info
VERSION 1.0.0
GIT_COMMIT e6903fa
...
Instead of a binary, a container image can be built using make image
. Note that currently only Linux image are supported and OS=linux
has to be set explicitly on other systems. Docker BuildKit is used for building images and needs to be installed and configured on the build system. Cross-compiling to other architectures is likewise controlled via the ARCH
option. Note that a corresponding builder needs to be configured in Docker BuildKit for this to work. Images for various architectures are provided for each release.
Running make
or make help
will output all available targets together with a description.
nagare media ingest
is mainly configured through a YAML configuration file. By default, a file named config.yaml
is searched for in the current directory and in /etc/nagare-media/ingest/
. The path can be overwritten with the -c, --config
CLI flag. The --dev
flag will activate a special developer mode (currently it only influences the logging output). The following additional flags are available:
$ ingest --help
Usage: ingest [options]
-c, --config string Path to the config file
--dev Run in developer mode
-h, --help Print this help message and exit
-l, --log-level string Log level ("debug", "info", "warn", "error", "panic", "fatal")
-V, --version Print the version number and exit
A usable example configuration is provided with config/samples/nagare-media/example.yaml
. The config/samples/nagare-media/full.yaml
configuration file lists and documents all available options and is meant as a reference.
We provide three examples for the DASH-IF Ingest protocol. To run these examples, either docker-compose
or FFmpeg needs to be installed locally. The examples are known to work with FFmpeg 5.0.1 running on Linux and macOS.
All example scenarios encode, package and send media with FFmpeg using the testsrc2
and sine
filters to generate test patterns for the video and audio tracks, respectively. Additionally, the wall-clock time at the encoding is burned into the video. We encode the video in 720p and 360p variants which is also burned into the video as text, respectively. The GOP length is set to 2 seconds with a constant frame rate of 25. Target bit rates are 2500 KBit/s and 600 KBit/s for the 720p and 360p video tracks, respectively. The audio track is encoded with constant 64 KBit/s.
The exact ffmpeg
command for the three scenarios is available in the scripts/tasks/run-*-ffmpeg
scripts and can be adjusted there.
This example uses the hls
FFmpeg muxer to output a DASH-IF Interface-2 compliant HLS stream to nagare media ingest
. A sliding window of five segments is used; old segments are deleted with HTTP DELETE
requests. This allows the usage of a mem
volume. The ingest starts after 10s and can be watched using the hls.js demo page.
# using Docker
$ docker-compose -f config/samples/nagare-media/docker-compose.hls-fmp4-ffmpeg.yaml up --build
# run locally (in two shells)
$ make clean build; bin/ingest-dev-* --dev -c config/samples/nagare-media/example.yaml
$ make run-hls-fmp4-ffmpeg
This example uses the dash
FFmpeg muxer to output a DASH-IF Interface-2 compliant LL-DASH stream to nagare media ingest
with a target latency of 3 seconds. The 8 second CMAF fragments are composed of 1 second chunks and are ingested using HTTP chunked transfer encoding. This allows for an early release from the encoder and immediate delivery to clients. The ingest starts after 10s and can be watched using the dash.js demo page.
# using Docker
$ docker-compose -f config/samples/nagare-media/docker-compose.ll-dash-ffmpeg.yaml up --build
# run locally (in two shells)
$ make clean build; bin/ingest-dev-* --dev -c config/samples/nagare-media/example.yaml
$ make run-ll-dash-ffmpeg
This example uses the normal mp4
FFmpeg muxer to output a DASH-IF Interface-1 compliant CMAF stream to nagare media ingest
. Each CMAF track is ingested using a long running HTTP POST
request with HTTP chunked transfer encoding. The 1 second CMAF chunks are split on the server side and an HLS manifest is generated. The ingest starts after 10s and can be watched using the hls.js demo page.
# using Docker
$ docker-compose -f config/samples/nagare-media/docker-compose.cmaf-long-upload-ffmpeg.yaml up --build
# run locally (in two shells)
$ make clean build; bin/ingest-dev-* --dev -c config/samples/nagare-media/example.yaml
$ make run-cmaf-long-upload-ffmpeg
This software was presented at the "13th ACM Multimedia Systems Conference, 2022 (MMSys '22)". Please cite this project in your research projects as follows:
Matthias Neugebauer. 2022. Nagare Media Ingest: A Server for Live CMAF Ingest Workflows. In 13th ACM Multimedia Systems Conference (MMSys ’22), June 14–17, 2022, Athlone, Ireland. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3524273.3532888
@inproceedings{10.1145/3524273.3532888,
author = {Neugebauer, Matthias},
title = {Nagare Media Ingest: A Server for Live CMAF Ingest Workflows},
year = {2022},
isbn = {9781450392839},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3524273.3532888},
doi = {10.1145/3524273.3532888},
abstract = {New media ingest protocols have been presented recently. SRT and RIST compete with old protocols such as RTMP while the DASH-IF specified an HTTP-based ingest protocol for CMAF formatted media that lends itself towards delivery protocols such as DASH and HLS. Additionally, use cases of media ingest workflows can vary widely. This makes implementing generic and flexible tools for ingest workflows a hard challenge. A monolithic approach limits adoption if the implementation does not fit the use case completely. We propose a new design for ingest servers that splits responsibilities into multiple components running concurrently. This design enables flexible ingest deployments as is discussed for various use cases. We have implemented this design in the open source software Nagare Media Ingest for the new DASH-IF ingest protocol.},
booktitle = {Proceedings of the 13th ACM Multimedia Systems Conference},
numpages = {6},
keywords = {video streaming, protocol, server, dash, cmaf},
location = {Athlone, Ireland},
series = {MMSys '22}
}
Apache 2.0 (c) nagare media authors