Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Create Label Error" when adding custom label on staging #1080

Open
JGreenlee opened this issue Jul 25, 2024 · 8 comments
Open

"Create Label Error" when adding custom label on staging #1080

JGreenlee opened this issue Jul 25, 2024 · 8 comments

Comments

@JGreenlee
Copy link

This error occurs when adding any custom label.

It is 100% reproducible using e-mission-phone (at master) and connecting to the staging server.
However, it is not reproducible when connecting to a local instance of e-mission-server (also at master); even with e-mission-phone still at master and same config (I was using stage-study). I am perplexed by this.

The error message would suggest malformed JSON, but I inspected what the UI is sending and it looks fine:
DEBUG:About to update custom label: {"key":"mode","old_label":"","new_label":"skateboard","is_new_label_must_added":true}

Googling the error message reveals that the error occurs in iOS native code. However, I couldn't find any more details in the loggerDB logs.

On the server logs, it appears that the /customlabel/update request is initiated and immediately exits:

2024-07-25T03:33:57.480Z | 2024-07-25 03:33:57,479:DEBUG:140584710653504:START POST /customlabel/update
-- | --
2024-07-25T03:33:57.480Z | 2024-07-25 03:33:57,480:DEBUG:140584710653504:END POST /customlabel/update 0.0003833770751953125

This is what /customlabel/update should be doing:

@post('/customlabel/update')
def updateUserCustomLabel():
  logging.debug("Called updateUserCustomLabel")
  updated_label = request.json['updated_label']
  user_uuid = getUUID(request)
  user = User.fromUUID(user_uuid)
  to_return = user.updateUserCustomLabel(updated_label)
  logging.debug("Successfully updated label label for user %s" % user_uuid)
  return { 'label' : to_return }

...but not even "Called updateUserCustomLabel" gets logged out.
@shankari
Copy link
Contributor

@JGreenlee interesting. given that:

  1. the local server works but stage doesn't
  2. we are seeing the message reach the stage server
  3. but not show up while processing the stage server

I think that the problem is with a server error, probably related to the database.
The error that we actually see is likely because we tried to parse the error on the server as JSON and failed.
e.g. https://stackoverflow.com/a/63679740

I checked the server logs, and it looks like the errors don't actually show up in the streamed console logs.
It might be easiest to print out the string in the native code before converting to JSON.

@JGreenlee LMK if you can do this, or whether I should (not sure if you are using the devapp or re-building)

@JGreenlee
Copy link
Author

@JGreenlee LMK if you can do this, or whether I should (not sure if you are using the devapp or re-building)

I think I can do that. I know how to build; I will just have to learn how to log from Objective-C

@JGreenlee
Copy link
Author

The native code is expecting a JSON string in the response, and it receives an HTML string instead. It appears to be the HTML of https://www.nrel.gov/transportation/openpath.html

I think this means the server is not recognizing /customlabel/update as a route and it's defaulting to something else.

@shankari Is there an extra step needed to configure new routes?

@shankari
Copy link
Contributor

We may want to leave that log in (catch the exception, log and then rethrow) to help catch such issues in the future.

Yes that is what it means.

Double-checking using curl, I get... 
$ curl https://openpath-stage.nrel.gov/api/usercache/get

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html>
        <head>
            <title>Error: 405 Method Not Allowed</title>
            <style type="text/css">
              html {background-color: #eee; font-family: sans-serif;}
              body {background-color: #fff; border: 1px solid #ddd;
                    padding: 15px; margin: 15px;}
              pre {background-color: #eee; border: 1px solid #ddd; padding: 5px;}
            </style>
        </head>
        <body>
            <h1>Error: 405 Method Not Allowed</h1>
            <p>Sorry, the requested URL <tt>&#039;https://openpath-stage.nrel.gov/usercache/get&#039;</tt>
               caused an error:</p>
            <pre>Method not allowed.</pre>
        </body>
    </html>

$ curl https://openpath-stage.nrel.gov/api/usercache/put

    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html>
        <head>
            <title>Error: 405 Method Not Allowed</title>
            <style type="text/css">
              html {background-color: #eee; font-family: sans-serif;}
              body {background-color: #fff; border: 1px solid #ddd;
                    padding: 15px; margin: 15px;}
              pre {background-color: #eee; border: 1px solid #ddd; padding: 5px;}
            </style>
        </head>
        <body>
            <h1>Error: 405 Method Not Allowed</h1>
            <p>Sorry, the requested URL <tt>&#039;https://openpath-stage.nrel.gov/usercache/put&#039;</tt>
               caused an error:</p>
            <pre>Method not allowed.</pre>
        </body>
    </html>

$ curl https://openpath-stage.nrel.gov/api/customlabel/update
$ curl https://openpath-stage.nrel.gov/api/customlabel/get
$

It certainly looks like the new API endpoints have not been deployed.
There should not be an extra step needed to configure the new routes, as we can see from the fact that the server works locally.

Double-checking deployment steps:

Screenshot 2024-07-25 at 12 10 11 PM - I deployed to jenkins with "Rebuild Apps" Screenshot 2024-07-25 at 12 28 12 PM

So we should have the new code deployed. I'm going to deploy from the image and see if I can verify. It would be good to also print out some kind of version information from the server at startup so we can verify which version is running.

@shankari
Copy link
Contributor

shankari commented Jul 26, 2024 •
edited
Loading

Ok so this is weird. I build the new image on Jul 23 and it was successful.
Screenshot 2024-07-25 at 6 01 51 PM

But I don't see the related image in the registry
Screenshot 2024-07-25 at 6 02 45 PM

Confirmed that the most recent image (the one from jul 18) does not have the changes.

$ docker run -p 8080:8080 -it <image from jul 18> /bin/bash
root@d246f9a5c2cb:/usr/src/app# grep -r customlabel emission/net/
root@d246f9a5c2cb:/usr/src/app#

So what was deployed? Was it actually the new image? @jgu2 can you take a look on whether this is related to your recent stage versus prod channel changes?

@shankari
Copy link
Contributor

Fixed by @jgu2, re-deployed on staging

@JGreenlee @louisg1337 assuming it does, can one of you submit a PR with the code that Jack added (catch the JSON format exception and rethrow with the raw response)? It will help debug such issues more easily in the future.

JGreenlee added a commit to JGreenlee/em-servercomm that referenced this issue Jul 26, 2024
@JGreenlee
if parsing response as JSON fails, include the raw response in error log
64195ea 
This was added in order to debug e-mission/e-mission-docs#1080
@JGreenlee JGreenlee mentioned this issue Jul 26, 2024
Merged
@JGreenlee
Copy link
Author

e-mission/cordova-server-communication#33

@shankari
Copy link
Contributor

Rebuilt but no new tag generated
Screenshot 2024-07-26 at 1 43 45 PM Screenshot 2024-07-26 at 1 44 23 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@shankari @JGreenlee