record rtmp with exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding #1858

wangyiedge · 2022-08-02T07:13:45Z

version: 1.5.7

I want to save the stream to MP4 file. I tried RTSP and m3u8, they are all success. But when using RTMP, I got an exception(avcodec_send_frame() error -541478725: Error sending a video frame for encoding). At first, I suspected that there was a problem with my RTMP stream, but when I changed a RTMP soure open on the Internet, the error stills there.

...
record! 82
record! 83
record! 84
2022-08-02 14:39:09.532  INFO 16831 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-08-02 14:39:09.579 ERROR 16831 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:774) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:755) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.2.jar:2.7.2]
	at cn.sibat.cvtest.CvTestApplication.main(CvTestApplication.java:10) [classes/:na]
Caused by: org.bytedeco.javacv.FFmpegFrameRecorder$Exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding. (For more details, make sure FFmpegLogCallback.set() has been called.)
	at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1056) ~[javacv-1.5.7.jar:1.5.7]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:961) ~[javacv-1.5.7.jar:1.5.7]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:954) ~[javacv-1.5.7.jar:1.5.7]
	at cn.sibat.cvtest.MainThread.run(MainThread.java:69) ~[classes/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
	... 5 common frames omitted

//        String url = "rtmp://10.0.24.23/live/2001";
String url = "rtmp://ns8.indexforce.com/home/mystream";
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(url);
grabber.start();

String localUrl = "/Users/yanwang/work/javacv.mp4";
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(localUrl, grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
recorder.setFormat("mp4");
recorder.setFrameRate(grabber.getFrameRate());
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setVideoBitrate(grabber.getVideoBitrate());
recorder.start();

LocalDateTime startTime = LocalDateTime.now();
int times = 0;
Frame frame = null;
while (startTime.plusSeconds(10).compareTo(LocalDateTime.now()) > 0) {
    if ((frame = grabber.grab()) != null) {
        recorder.record(frame);
        times++;
        System.out.println("record! "+times);
    }
}
recorder.stop();
grabber.stop();
System.out.println("success");

saudet · 2022-08-02T07:31:35Z

Please make sure to call FFmpegLogCallback.set() to get more detailed messages on the console.

wangyiedge · 2022-08-02T07:51:17Z

Actually, the code contains FFmpegLogCallback.set()
add the full amount of logs as follows, What is omitted in the middle is that every time I call record (), Print
Call about 100 times, the exception will throws

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.7.2)

2022-08-02 15:38:44.645  INFO 46512 --- [           main] cn.sibat.cvtest.CvTestApplication        : Starting CvTestApplication using Java 1.8.0_172 on YandeMacBook-Pro.local with PID 46512 (/Users/yanwang/gitProjects/study/cv-test/target/classes started by yanwang in /Users/yanwang/gitProjects/study/cv-test)
2022-08-02 15:38:44.648  INFO 46512 --- [           main] cn.sibat.cvtest.CvTestApplication        : No active profile set, falling back to 1 default profile: "default"
2022-08-02 15:38:45.486  INFO 46512 --- [           main] cn.sibat.cvtest.CvTestApplication        : Started CvTestApplication in 1.506 seconds (JVM running for 2.44)
Info: Input #0, flv, from 'rtmp://ns8.indexforce.com/home/mystream':

Info:   Metadata:

Info:     |RtmpSampleAccess: 
Info: false
Info: 

Info:     fileSize        : 
Info: 0
Info: 

Info:     audiochannels   : 
Info: 2
Info: 

Info:     2.1             : 
Info: false
Info: 

Info:     3.1             : 
Info: false
Info: 

Info:     4.0             : 
Info: false
Info: 

Info:     4.1             : 
Info: false
Info: 

Info:     5.1             : 
Info: false
Info: 

Info:     7.1             : 
Info: false
Info: 

Info:     encoder         : 
Info: obs-output module (libobs version 27.2.1)
Info: 

Info:   Duration: 
Info: 00:00:00.00
Info: , start: 
Info: 0.000000
Info: , bitrate: 
Info: N/A
Info: 

Info:   Stream #0:0
Info: : Data: none
Info: 

Info:   Stream #0:1
Info: : Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1048 kb/s
Info: , 
Info: 15 fps, 
Info: 15 tbr, 
Info: 1k tbn
Info: 

Info:   Stream #0:2
Info: : Audio: aac (LC), 48000 Hz, stereo, fltp, 65 kb/s
Info: 

Warning: [libopenh264 @ 0x7fb768086280] Slice count will be set automatically

Warning: [libopenh264 @ 0x7fb768086280] [OpenH264] this = 0x0x7fb765ce7a50, Warning:bEnableFrameSkip = 0,bitrate can't be controlled for RC_QUALITY_MODE,RC_BITRATE_MODE and RC_TIMESTAMP_MODE without enabling skip frame.

Warning: [libopenh264 @ 0x7fb768086280] [OpenH264] this = 0x0x7fb765ce7a50, Warning:Change QP Range from(0,51) to (12,42)

Info: Output #0, mp4, to '/Users/yanwang/work/javacv.mp4':

Info:   Metadata:

Info:     encoder         : 
Info: Lavf59.16.100
Info: 

Info:   Stream #0:0
Info: : Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, q=2-31, 1048 kb/s
Info: , 
Info: 15 fps, 
Info: 15360 tbn
Info: 

Info:   Stream #0:1
Info: : Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
Info: 

record! 1
record! 2
record! 3
record! 4
record! 5
record! 6
record! 7
record! 8
record! 9
record! 10

...

record! 121
record! 122
record! 123
record! 124
record! 125
record! 126
record! 127
record! 128
record! 129
2022-08-02 15:38:55.047  INFO 46512 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-08-02 15:38:55.098 ERROR 46512 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:774) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:755) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.2.jar:2.7.2]
	at cn.sibat.cvtest.CvTestApplication.main(CvTestApplication.java:10) [classes/:na]
Caused by: org.bytedeco.javacv.FFmpegFrameRecorder$Exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding. (For more details, make sure FFmpegLogCallback.set() has been called.)
	at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1056) ~[javacv-1.5.7.jar:1.5.7]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:961) ~[javacv-1.5.7.jar:1.5.7]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:954) ~[javacv-1.5.7.jar:1.5.7]
	at cn.sibat.cvtest.MainThread.run(MainThread.java:69) ~[classes/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
	... 5 common frames omitted


Process finished with exit code 1

saudet · 2022-08-02T07:52:36Z

Please try with libx264 instead of libopenh264.

wangyiedge · 2022-08-02T08:09:14Z

I replace recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264) with recorder.setVideoCodecName("libx264") is that right?
Got same exception, logs as follow

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.7.2)

2022-08-02 16:00:39.464  INFO 57202 --- [           main] cn.sibat.cvtest.CvTestApplication        : Starting CvTestApplication using Java 1.8.0_172 on YandeMacBook-Pro.local with PID 57202 (/Users/yanwang/gitProjects/study/cv-test/target/classes started by yanwang in /Users/yanwang/gitProjects/study/cv-test)
2022-08-02 16:00:39.466  INFO 57202 --- [           main] cn.sibat.cvtest.CvTestApplication        : No active profile set, falling back to 1 default profile: "default"
2022-08-02 16:00:40.494  INFO 57202 --- [           main] cn.sibat.cvtest.CvTestApplication        : Started CvTestApplication in 1.667 seconds (JVM running for 2.786)
Info: Input #0, flv, from 'rtmp://ns8.indexforce.com/home/mystream':

Info:   Metadata:

Info:     |RtmpSampleAccess: 
Info: false
Info: 

Info:     fileSize        : 
Info: 0
Info: 

Info:     audiochannels   : 
Info: 2
Info: 

Info:     2.1             : 
Info: false
Info: 

Info:     3.1             : 
Info: false
Info: 

Info:     4.0             : 
Info: false
Info: 

Info:     4.1             : 
Info: false
Info: 

Info:     5.1             : 
Info: false
Info: 

Info:     7.1             : 
Info: false
Info: 

Info:     encoder         : 
Info: obs-output module (libobs version 27.2.1)
Info: 

Info:   Duration: 
Info: 00:00:00.00
Info: , start: 
Info: 0.000000
Info: , bitrate: 
Info: N/A
Info: 

Info:   Stream #0:0
Info: : Data: none
Info: 

Info:   Stream #0:1
Info: : Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 1048 kb/s
Info: , 
Info: 15 fps, 
Info: 15 tbr, 
Info: 1k tbn
Info: 

Info:   Stream #0:2
Info: : Audio: aac (LC), 48000 Hz, stereo, fltp, 65 kb/s
Info: 

Info: Output #0, mp4, to '/Users/yanwang/work/javacv.mp4':

Info:   Metadata:

Info:     encoder         : 
Info: Lavf59.16.100
Info: 

Info:   Stream #0:0
Info: : Video: mpeg4 (mp4v / 0x7634706D), yuv420p, 1920x1080, q=2-31, 1048 kb/s
Info: , 
Info: 15 fps, 
Info: 15360 tbn
Info: 

Info:   Stream #0:1
Info: : Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
Info: 

record! 1
record! 2
record! 3
...
record! 74
record! 75
2022-08-02 16:00:49.183  INFO 57202 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-08-02 16:00:49.226 ERROR 57202 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Failed to execute CommandLineRunner
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:774) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:755) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.2.jar:2.7.2]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.2.jar:2.7.2]
	at cn.sibat.cvtest.CvTestApplication.main(CvTestApplication.java:10) [classes/:na]
Caused by: org.bytedeco.javacv.FFmpegFrameRecorder$Exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding. (For more details, make sure FFmpegLogCallback.set() has been called.)
	at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1056) ~[javacv-1.5.7.jar:1.5.7]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:961) ~[javacv-1.5.7.jar:1.5.7]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:954) ~[javacv-1.5.7.jar:1.5.7]
	at cn.sibat.cvtest.MainThread.run(MainThread.java:70) ~[classes/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
	... 5 common frames omitted


Process finished with exit code 1

saudet · 2022-08-02T08:22:48Z

You'll need to add ffmpeg-platform-gpl in your dependencies as shown in the README.md file:
https://github.com/bytedeco/javacv/#sample-usage

wangyiedge · 2022-08-02T09:40:52Z

don't work, same exception

with add dependency in pom.xml

<dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>ffmpeg-platform-gpl</artifactId>
    <version>5.0-1.5.7</version>
</dependency>

compare the logs before and after, the following lines are new.

...
Info:   Stream #0:2
Info: : Audio: aac (LC), 48000 Hz, stereo, fltp, 65 kb/s
Info: 

Info: [libx264 @ 0x7fb0550a6700] using cpu capabilities: none!

Info: [libx264 @ 0x7fb0550a6700] profile High, level 4.0, 4:2:0, 8-bit

Info: [libx264 @ 0x7fb0550a6700] 264 - core 163 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - 
http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 
chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 
constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 
keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=1048 ratetol=1.0 qcomp=0.60 
qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00

Info: Output #0, mp4, to '/Users/yanwang/work/javacv.mp4':
...

I try to reduce the version(without add ⬆️dependency)，got some information, hope can help

version 1.5.7 -> 1.5.6 :

Nothing Changed

version 1.5.7 -> 1.5.4 :

Got new Exception,(seem 1.5.4 default use libx264?)

...
record! 32
record! 33
record! 34
Error: [libx264 @ 0x7f89c8ab0a00] lookahead thread is already stopped
...

Caused by: org.bytedeco.javacv.FrameRecorder$Exception: avcodec_encode_video2() error -542398533: Could not encode video packet.
	at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1045) ~[javacv-1.5.4.jar:1.5.4]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:948) ~[javacv-1.5.4.jar:1.5.4]
	at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:941) ~[javacv-1.5.4.jar:1.5.4]
	at cn.sibat.cvtest.MainThread.run(MainThread.java:71) ~[classes/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:771) [spring-boot-2.7.2.jar:2.7.2]
	... 5 common frames omitted

version 1.5.7 -> 1.5.5 :

No error, no exception, it worked!!!

So...why？

saudet · 2022-08-02T10:18:14Z

Those errors probably mean "end of file". It's most likely just a coincidence that it works sometimes. It sounds like a problem with the network.

wangyiedge · 2022-08-03T03:43:02Z

I tried many times, this bug always reappear. seems not like a network problem.
and I found an old issues in 1.5.4，guess thats why 1.5.5 can work:
#1563

so, Is it due to other reasons that similar bugs are reintroduced in later versions?

saudet · 2022-08-04T03:01:13Z

Good point, this workaround probably got lost when upgrading to FFmpeg 5.0 where those deprecated functions are no longer available.

wangyiedge · 2022-08-04T10:40:59Z

I found that, the frame from the first 'grabber.grab()' , always 'DATA' type. when ignore it, I got correct result, like:

Frame frame = grabber.grab();
System.out.println("ignore the first frame，type: " + frame.getTypes()); //[DATA]
while ((frame = grabber.grab()) != null) {
    recorder.record(frame);
}

Although I don't know the reason, it may be helpful to others

…codec on data frames (issue #1858)

saudet · 2022-08-04T13:10:11Z

I think I understand what's going on. This should be fixed with commit 32f5fed.
Please give it a try with the snapshots: http://bytedeco.org/builds/

wangyiedge · 2022-08-05T01:51:27Z

it works, thank you @saudet

saudet · 2022-11-03T03:46:13Z

The fix has been released with JavaCV 1.5.8. Thanks for reporting!

saudet added help wanted question labels Aug 2, 2022

saudet added a commit that referenced this issue Aug 4, 2022

* Fix FFmpegFrameRecorder.record() incorrectly flushing the video …

32f5fed

…codec on data frames (issue #1858)

saudet added bug and removed help wanted question labels Aug 4, 2022

saudet mentioned this issue Aug 8, 2022

crash: libavformat.so.58+0x1513ee #1852

Closed

saudet closed this as completed Nov 3, 2022

saudet mentioned this issue Jan 8, 2023

Ignore non-video and non-audio frames in FFmpegFrameRecorder.record #1695

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

record rtmp with exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding #1858

record rtmp with exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding #1858

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 3, 2022

saudet commented Aug 4, 2022

wangyiedge commented Aug 4, 2022

saudet commented Aug 4, 2022

wangyiedge commented Aug 5, 2022

saudet commented Nov 3, 2022

record rtmp with exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding #1858

record rtmp with exception: avcodec_send_frame() error -541478725: Error sending a video frame for encoding #1858

Comments

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 2, 2022

saudet commented Aug 2, 2022

wangyiedge commented Aug 3, 2022

saudet commented Aug 4, 2022

wangyiedge commented Aug 4, 2022

saudet commented Aug 4, 2022

wangyiedge commented Aug 5, 2022

saudet commented Nov 3, 2022