Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image Issue on Nginx #56038

Closed
Iulian-Dragomirescu opened this issue Sep 26, 2023 · 92 comments
Closed

Image Issue on Nginx #56038

Iulian-Dragomirescu opened this issue Sep 26, 2023 · 92 comments
Labels
linear: next Confirmed issue that is tracked by the Next.js team. locked

Comments

@Iulian-Dragomirescu
Copy link

Iulian-Dragomirescu commented Sep 26, 2023

HI,
Image works in development and production (only locally, not remotely), but in nginx I get this error (from nginx console):

upstream image response failed for /assets/icons/gdpr.png TypeError: fetch failed
cause: ConnectTimeoutError: Connect Timeout Error

my folders
public/assets/icons/gdpr.png

        <Image
          onError={(e) => {
            console.error("Error loading image:", e);
          }}
          src="/assets/icons/gdpr.png"
          className="rounded-sm"
          alt="gdpr"
          width={100}
          height={100}
        />

i try to convert the image in webp.. same error
restart server.. same

and this:
"url" parameter is valid but upstream response is invalid
i receive from chrome exemple.com/_next/image?url=%2Fassets%2Ficons%2Fgdpr.png&w=256&q=75

image

Any ideea?

Thanks!

Additional information

Operating System:
  Platform: win32
  Arch: x64
  Version: Windows 10 Home
Binaries:
  Node: 18.17.1
  npm: N/A
  Yarn: N/A
  pnpm: N/A
Relevant Packages:
  next: 13.5.3
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: N/A
Next.js Config:
  output: N/A

NEXT-1680

@Iulian-Dragomirescu Iulian-Dragomirescu changed the title ### Summary Image Issue on Nginx Sep 26, 2023
@dennisofficial
Copy link

Wow, same here:
image
image

Use to work fine before.

@Iulian-Dragomirescu
Copy link
Author

i get the same error but i solve with:

image

after this i get new error =))
cause: ConnectTimeoutError: Connect Timeout Error

@igeligel
Copy link

Reporting the same: https://gist.github.com/igeligel/13473e8702239fa17ec83a89a30f66f8

Some more information:

I think it might be related to this: #55775

Picture was always on the server:

chrome_r62GNd0AIF

@amit548
Copy link

amit548 commented Sep 26, 2023

image

image

Same error, some images works well but some image not loaded and throw this error '"url" parameter is valid but upstream response is invalid'. I am using nginx web server.

@Paget96
Copy link

Paget96 commented Sep 26, 2023

Remotely it will only work if you put your server IP instead of the domain.

@amit548
Copy link

amit548 commented Sep 26, 2023

Remotely it will only work if you put your server IP instead of the domain.

Yes you right, if we put server IP then it works well, we spend lots of time with this issue.

@Iulian-Dragomirescu
Copy link
Author

I downgrade to 13.5.2, now error is "The requested resource isn't a valid image for /assets/icons/gdpr.png received text/html; charset=utf-8"

@amit548
Copy link

amit548 commented Sep 26, 2023

We did downgrade to 13.5.2 and works well for now :)

@aslakson
Copy link

Experiencing the same issue. I will backup to 13.5.2 in the mean time.

@cduff
Copy link

cduff commented Sep 27, 2023

Experiencing same issue with 13.5.3. In my case running Next.js in Docker container (http) then served over AWS CloudFront (https). Reverting to 13.5.2 fixes it for now.

@garryxiao
Copy link

Same issue with 13.5.3. Downgrade to 13.5.2 works as expected.

@Iulian-Dragomirescu
Copy link
Author

Iulian-Dragomirescu commented Sep 27, 2023

I managed to find the solution for version 13.5.3 as well. || For Node.js server **

const port = process.env.PORT || 3000;  // this line

// Next js init
const dev = process.env.NODE_ENV !== "production";
const server = next({ dev, dir: process.cwd(), port }); // need to put "port" here
const handle = server.getRequestHandler();

@Jerome1337
Copy link

I have the same problem on next 13.5.3 with the following error on every public images "url" parameter is valid but upstream response is invalid.
Screenshot 2023-09-27 at 10 53 39 AM

fixing to the 13.5.2 for now.

@johansunden
Copy link

Got this issue too when I upgraded from 13.4.19 to v13.5.3, so had to revert that upgrade. Will probably upgrade to v15.2 based on previous comments in this thread.

@advancingu
Copy link

Fyi, some of the issues reported here should be fixed by #55988

@gregorybolkenstijn
Copy link

I can confirm the issue on 13.5.3 and that downgrading to 13.5.2 resolved the issue with "url" parameter is valid but upstream response is invalid.

@benfavre

This comment was marked as off-topic.

@FlorianLeChat
Copy link

FlorianLeChat commented Oct 3, 2023

It's crazy how Next.js is pushing me more and more to stop putting so much effort into making the latest stable updates if it's only to get these kinds of bugs, I understand why many people have had a bad experience with this framework in the past or recently, it's a shame. Every update is like Russian roulette to see if it hasn't introduced a new bug.

@breakerh
Copy link

breakerh commented Oct 3, 2023

upgrading to 15.5.4 fixes this issue for us.

Edit: nextjs app setup with api and frontend routes, typoscript, deployed on digital ocean app platform.

@DOZBORNE
Copy link

DOZBORNE commented Oct 3, 2023

Upgrading to 13.5.4 does not work for us. We're getting "url" parameter is valid but upstream response is invalid." On the server side, we get an error like so. This is not working for us unless we rollback our versions.
image

@cduff
Copy link

cduff commented Oct 3, 2023

Experiencing same issue with 13.5.3. In my case running Next.js in Docker container (http) then served over AWS CloudFront (https). Reverting to 13.5.2 fixes it for now.

Fixed in 13.5.4 for me.

@pascalstr
Copy link

13.5.4 fixes this issue for us.

@amit548
Copy link

amit548 commented Oct 4, 2023

13.5.4 works for me ❤️

@aslakson
Copy link

aslakson commented Oct 4, 2023

13.5.4 does not fix the issue for me - receiving the following errors:

upstream image response failed for /_next/static/media/logo-fff.5c2fe673.png TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11576:11)
    at async invokeRequest (/Users/[redacted]/node_modules/next/dist/server/lib/server-ipc/invoke-request.js:16:12)
    at async /Users/[redacted]/node_modules/next/dist/server/next-server.js:362:35
    at async imageOptimizer (/Users/[redacted]/node_modules/next/dist/server/image-optimizer.js:537:13)
    at async cacheEntry.imageResponseCache.get.incrementalCache (/Users/[redacted]/node_modules/next/dist/server/next-server.js:517:61)
    at async /Users/[redacted]/node_modules/next/dist/server/response-cache/index.js:102:36 {
  cause: InvalidArgumentError: invalid connection header
      at processHeader (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:102994)
      at new Request (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:100295)
      at [dispatch] (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:64352)
      at Intercept (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:2:93153)
      at [Intercepted Dispatch] (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:115359)
      at Client.dispatch (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:115591)
      at [dispatch] (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:2:264035)
      at [Intercepted Dispatch] (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:115258)
      at Pool.dispatch (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:115591)
      at [dispatch] (/Users/[redacted]/node_modules/next/dist/compiled/undici/index.js:1:29874) {
    code: 'UND_ERR_INVALID_ARG'

Using app and pages directory behind NGINX. If I'm supposed to be adding some sort of header in my NGINX config, a nudge in the right direction would be greatly appreciated.

@torchsmith
Copy link

I'm getting identical error as @aslakson

I'm only on app dir behind NGINX though, not pages.

@dennisofficial
Copy link

dennisofficial commented Oct 4, 2023

Hey everyone!! Hope my message can help a lot of you guys out.

My guess, there was a bug introduced from an update that is causing all of this. I've spent days trying to fix it, and the only fix is downgrading.

MY ISSUES:
Internal images are not showing in production (It didn't matter the method, public folder, import method, or absolute path), external images work fine.

NOTES:
13.5.4 did not work for me.

I've gotten all the errors I see on this thread.

As for my setup if this is something you have similar:

  • SSL with Cloudflare
  • Remote server with docker containers running the NextJS server exposing to port 3000.
  • NGNIX acting as a proxy and load balancer listening to requests on port 80 and 443 and forwarding that to port 3000.

package.json:

"sharp": "^0.32.6",
"react": "18.2.0",
"next": "13.4.12",

Server Versions

nginx version: nginx/1.18.0 (Ubuntu)
Docker version 24.0.5, build ced0996

NGINX CONFIG:

server {
    listen 80;
    listen 443 ssl;

    server_name your_domain_here.com www.your_domain_here.com;

    ssl_certificate /etc/nginx/ssl/server.crt;
    ssl_certificate_key /etc/nginx/ssl/server.key;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP $proxy_protocol_addr;
        proxy_set_header X-Forwarded-For $proxy_protocol_addr;
        proxy_set_header Host $host;

        proxy_headers_hash_max_size 512;
        proxy_headers_hash_bucket_size 128;

        error_page 502 503 504 /maintenance.html;
        proxy_intercept_errors on;
    }

    location /backend {
        proxy_pass http://127.0.0.1:4000;
        proxy_redirect off;
    }
}

@huozhi huozhi closed this as completed Oct 16, 2023
styfle added a commit that referenced this issue Oct 16, 2023
This PR adds a test to ensure we don't regress again when using proxies
in front of Next.js with `connection: 'upgrade'`.

- Regression introduced in #56226
- Issue reported in #56038
- Rolled back in #56836
@ShreshthTiwari
Copy link

ShreshthTiwari commented Oct 17, 2023

We released a fix for this in 13.5.5, please upgrade to the new version

Updated NextJS to 13.5.5 and now receiving this error

TypeError: e.getSetCookie is not a function
  at new u (/home/container/.next/server/chunks/955.js:1:3389)
  at new NextResponse (/home/container/.next/server/chunks/955.js:1:9744)
  at POST (/home/container/.next/server/app/api/properties/list/route.js:1:3983)
  at process.processTicksAndRejections (node:internal/process/task_quesues:95:5)
  at async /home/container/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:14:39709

this is my package.js dependencies

"dependencies": {
    "@types/nodemailer": "^6.4.11",
    "@types/react": "^18.2.28",
    "@types/react-dom": "^18.2.13",
    "axios": "^1.5.1",
    "axios-retry": "^3.8.0",
    "eslint": "^8.51.0",
    "eslint-config-next": "^13.5.5",
    "mysql2": "^3.6.2",
    "next": "^13.5.5",
    "nodemailer": "^6.9.6",
    "razorpay": "^2.9.2",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-icons": "^4.11.0",
    "react-simple-pull-to-refresh": "^1.3.3",
    "sharp": "^0.32.6",
    "typescript": "^5.2.2"
  }

I tried rolling back to 13.5.4 but now this error is still showing up

TypeError: e.getSetCookie is not a function
  at new u (/home/container/.next/server/chunks/955.js:1:3389)
  at new NextResponse (/home/container/.next/server/chunks/955.js:1:9744)
  at POST (/home/container/.next/server/app/api/properties/list/route.js:1:3983)
  at process.processTicksAndRejections (node:internal/process/task_quesues:95:5)
  at async /home/container/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:14:39709

@dennisofficial
Copy link

We released a fix for this in 13.5.5, please upgrade to the new version

Updated NextJS to 13.5.5 and now receiving this error

this is my package.js dependencies


"dependencies": {

    "@types/nodemailer": "^6.4.11",

    "@types/react": "^18.2.24",

    "@types/react-dom": "^18.2.8",

    "@types/react-helmet": "^6.1.7",

    "axios": "^1.5.1",

    "axios-retry": "^3.8.0",

    "eslint": "^8.50.0",

    "eslint-config-next": "^13.5.4",

    "mysql2": "^3.6.1",

    "next": "^13.5.5",

    "nodemailer": "^6.9.5",

    "razorpay": "^2.9.2",

    "react": "^18.2.0",

    "react-dom": "^18.2.0",

    "react-icons": "^4.11.0",

    "react-simple-pull-to-refresh": "^1.3.3",

    "sharp": "^0.32.6",

    "typescript": "^5.2.2"

  }

I tried rolling back to 13.5.4 but now this error is still showing up


TypeError: e.getSetCookie is not a function

  at new u (/home/container/.next/server/chunks/955.js:1:3389)

  at new NextResponse (/home/container/.next/server/chunks/955.js:1:9744)

  at POST (/home/container/.next/server/app/api/properties/list/route.js:1:3983)

  at process.processTicksAndRejections (node:internal/process/task_quesues:95:5)

  at async /home/container/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:14:39709

Try deleting your .next folder. That usually fixes that.

@ShreshthTiwari
Copy link

ShreshthTiwari commented Oct 17, 2023

We released a fix for this in 13.5.5, please upgrade to the new version

Updated NextJS to 13.5.5 and now receiving this error
this is my package.js dependencies

"dependencies": {
    "@types/nodemailer": "^6.4.11",
    "@types/react": "^18.2.28",
    "@types/react-dom": "^18.2.13",
    "axios": "^1.5.1",
    "axios-retry": "^3.8.0",
    "eslint": "^8.51.0",
    "eslint-config-next": "^13.5.5",
    "mysql2": "^3.6.2",
    "next": "^13.5.5",
    "nodemailer": "^6.9.6",
    "razorpay": "^2.9.2",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-icons": "^4.11.0",
    "react-simple-pull-to-refresh": "^1.3.3",
    "sharp": "^0.32.6",
    "typescript": "^5.2.2"
  }

I tried rolling back to 13.5.4 but now this error is still showing up


TypeError: e.getSetCookie is not a function

  at new u (/home/container/.next/server/chunks/955.js:1:3389)

  at new NextResponse (/home/container/.next/server/chunks/955.js:1:9744)

  at POST (/home/container/.next/server/app/api/properties/list/route.js:1:3983)

  at process.processTicksAndRejections (node:internal/process/task_quesues:95:5)

  at async /home/container/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:14:39709

Try deleting your .next folder. That usually fixes that.

deleted node_modules, package-lock.json and .next folder but the issue isnt fixing

image

@MixMe
Copy link

MixMe commented Oct 17, 2023

Same problem, 13.5.5:

Screenshot 2023-10-17 at 11 25 16

@ShreshthTiwari
Copy link

It looks like this issue is occuring only when the app is running on a docker container.
I tried both ways-

changing port
running on root instead of docker

and the second method of running on root works but i dont wanna run it on root. i need to run it on a docker container please fix it.

@dennisofficial
Copy link

It looks like this issue is occuring only when the app is running on a docker container.

I tried both ways-

changing port

running on root instead of docker

and the second method of running on root works but i dont wanna run it on root. i need to run it on a docker container please fix it.

It'll honestly be best to downgrade and wait for a real stable release. I've been watching this thread as soon as the bug was found, and been waiting until this issue is closed so I can upgrade peacefully.

@huozhi
Copy link
Member

huozhi commented Oct 17, 2023

@dennisofficial @ShreshthTiwari @MixMe Can you file a new issue with reproduction? It's quite impossible for us to see what triggered that error only with the error trace here.

@dennisofficial
Copy link

#56038 (comment)
I don't have time to make a reproduction because of course it'll work fine in a standalone NextJS environment, without any third party servers (NGINX, Docker, etc)

The point of the issue is that before images would work fine in production environments either with a Docker Container, or PM2, or proxying with NGINX, etc. Using a trusted SSL, or not, or proxying a domain through CloudFlare, etc.

The problem isn't that NextJS broke in its self, the problem is there was a release that made it no longer compatible with the third party services. In my case, the images were working fine, and we updated NextJS. We tried removing NGINX that didn't solve it, however, we turned off CloudFlare SSL proxy, and now the images work fine. So when I access my remote server via direct-IP, the images load fine, once I switch to HTTPS, I get the "url" parameter is valid but upstream response is invalid error. The only way I fixed it, was by rolling back to my previous version.

To note, even with the latest update, it works fine in prod locally, but not in the cloud.

The only way we can reproduce this, is if you host a remote server, integrate CloudFlare, and pay for these services that makes a production environment.

@dennisofficial
Copy link

dennisofficial commented Oct 17, 2023

Or if there were implementations made that changed the way HTTPS requests are handled, maybe give us a documentation on what headers/etc is required to have HTTPS work in a production environment.

@Stevemoretz
Copy link

Stevemoretz commented Oct 17, 2023

@huozhi You can put the following code in your middleware, and you can test it no need for paying for anything or hosting anything in production I guess.

export async function middleware(request: NextRequest) {
    if (
        request.nextUrl.pathname.startsWith("/_next/image") &&
        request.nextUrl.searchParams.get("url")?.startsWith("/storage")
    ) {
        return NextResponse.rewrite(
            new URL(
                "https://dreamworldestate.com/wp-content/uploads/2022/10/app-store-apple-download-logo-png-favpng-2BMS9KidgmX2fHMtAhjpVHway-removebg-preview-1.png"
            ),
        );
    }
    
    return NextResponse.next();
}

Then add a image component in your UI:

<Image 
   src="/storage/does_not_matter_whatever_here"
   width={100}
   height={100}
/>

@dennisofficial
Copy link

dennisofficial commented Oct 17, 2023

@Stevemoretz, @huozhi

On my next deployment, I'll share a link. Either way, this will be a good time to test the latest version, I haven't tried it personally from all the comments I've read.

@Stevemoretz
Copy link

Stevemoretz commented Oct 17, 2023

It's not that hard to test it @dennisofficial if you can provide a picture URL on your website behind Cloudflare.

@huozhi Then you can put the following code in your middleware, and you can test it no need for paying for anything or hosting anything in production I guess.

export async function middleware(request: NextRequest) {
    if (
        request.nextUrl.pathname.startsWith("/_next/image") &&
        request.nextUrl.searchParams.get("url")?.startsWith("/storage")
    ) {
        return NextResponse.rewrite(
            new URL(
                "https://dreamworldestate.com/wp-content/uploads/2022/10/app-store-apple-download-logo-png-favpng-2BMS9KidgmX2fHMtAhjpVHway-removebg-preview-1.png"
            ),
        );
    }
    
    return NextResponse.next();
}

Then add a image component in your UI:

<Image 
   src="/storage/does_not_matter_whatever_here"
   width={100}
   height={100}
/>

@huozhi That's okay I found an image from an old website I did a while ago which is hosted behind cloudflare and updated my last message, you can try it now :)

@dennisofficial
Copy link

dennisofficial commented Oct 17, 2023

@Stevemoretz

Is that image hosted on your NextJS server? Just a note, I'm having issues with internal images, not external

Also, your code is redundant. request.url has no effect on new URL()

@grzegorzpokorski
Copy link

@dennisofficial @ShreshthTiwari @MixMe Can you file a new issue with reproduction? It's quite impossible for us to see what triggered that error only with the error trace here.

@huozhi I created issue: #56947

@dennisofficial
Copy link

grzegorzpokorski

We are having a different issue. I'm having the issue with "url" parameter is valid but upstream response is invalid.

@DOZBORNE
Copy link

Still having problems with this in production. Same error "url" parameter is valid but upstream response is invalid when fetching local images.

Upgraded to 13.5.5. No change in behavior whatsoever.

@dennisofficial
Copy link

Still having problems with this in production. Same error "url" parameter is valid but upstream response is invalid when fetching local images.

Upgraded to 13.5.5. No change in behavior whatsoever.

Damn, now I'm worried about checking 13.5.5, my CI/CD is very slow, it takes forever to redeploy.

@dennisofficial
Copy link

Still having problems with this in production. Same error "url" parameter is valid but upstream response is invalid when fetching local images.

Upgraded to 13.5.5. No change in behavior whatsoever.

What setup are you using? NGINX, Docker, any proxies?

@Stevemoretz
Copy link

@Stevemoretz

Is that image hosted on your NextJS server? Just a note, I'm having issues with internal images, not external

Also, your code is redundant. request.url has no effect on new URL()

@dennisofficial It's behind cloudflare but with an Apache server.

@DOZBORNE
Copy link

Just docker and then distributed on K8s. No virtual load balancers or anything like that

@Stevemoretz
Copy link

Still having problems with this in production. Same error "url" parameter is valid but upstream response is invalid when fetching local images.
Upgraded to 13.5.5. No change in behavior whatsoever.

Damn, now I'm worried about checking 13.5.5, my CI/CD is very slow, it takes forever to redeploy.

Try my suggestion in local if you have time for it, it will possibly produce the error then you can share the repro, I would but it takes a billion years on my machine to install a new version.

It most likely doesn't matter what your server is when it's behind Cloudflare anyway...

@huozhi
Copy link
Member

huozhi commented Oct 17, 2023

For getSetCookie issue: some node version is missing this method and upgrade to newer nodejs version works for me. You can try to upgrade the nodejs version to see if this is still present

#56947 (comment)

@ShreshthTiwari
Copy link

ShreshthTiwari commented Oct 18, 2023

I am using nodejs version 19 in docker.

Ah ok thanks this issue is occuring with Nodejs v19, do not use it only use the nodejs 18(hydrogen) LTS latest.

@Trickster811
Copy link

image

image

Same error, some images works well but some image not loaded and throw this error '"url" parameter is valid but upstream response is invalid'. I am using nginx web server.

Its works fine for me since i upgrade to 13.5.5 version

Copy link
Contributor

github-actions bot commented Nov 2, 2023

This closed issue has been automatically locked because it had no new activity for 2 weeks. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

@github-actions github-actions bot added the locked label Nov 2, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
linear: next Confirmed issue that is tracked by the Next.js team. locked
Projects
None yet
Development

No branches or pull requests