"Error while fetching from Google Scholar" and potential solution to the problem

[AEgit]

Yesterday I ran into the following error message when trying to use the inbuilt Web search of Jabref (Google Scholar was selected):

"Error while fetching from Google Scholar"

This problem has already been mentioned here:
http://discourse.jabref.org/t/error-while-fetching-from-google-scholar/181

It has been suggested that Google is blocking access due to "automatic crawling". Well, I haven't done any automatic crawling, but I guess I'm a heavy "manual" user of Google Scholar which also seems to cause such blocking. I was able to use Google Scholar from within any browser, just Jabref's Web search didn't work anymore.
I just wanted to share the solution I came up with, in case someone else has this problem as well.

I tried to use the Google Scholar Web Search of Jabref on different operating systems (different Linux distributions, Windows 10) and using different IP adresses (by using a VPN and physically moving myself to another place with a different network). None of these attempts solved the problem.

What solved the problem was to change the MAC address of my PC. Apparently Google blocks the search requests based on the MAC address. I tried that first in my virtual machines where it worked flawlessly. Funnily enough, after having used Google Scholar with a different MAC address under the same IP address, the Web search would also work again on my main PC without having to change its MAC address. I don't know whether that was due to my actions or whether Google finally decided to unblock me. Anyway, it works now.

TL;DR: Just change the MAC address and Google Scholar should work again from within Jabref.

[lenhard]

Good to know, thanks for the info!

Changing the MAC address will be impossible (or rather not desirable) from within JabRef, though.

Refs # #1886

[AEgit]

Shall I close this one? I just wanted to let people know about a potential solution to the problem (if they ever come across it).

[lenhard]

We should document your solution to preserve it. Closed issues are sometimes hard to find via search engines. Let us leave this issue open until we have added one or two sentences to http://help.jabref.org/en/GoogleScholar That way, your solution will really be preserved.

[AEgit]

Ok.

[lenhard]

Fixed in JabRef/help.jabref.org@d5f5725

[alephreish]

For the record: I don't think there is any way for google to know the user's MAC address. Changing it locally might trigger something else on some systems, but it is a hack that should be avoided as much as possible. Google relies on cookies and IP address only, as far as I can tell from my experience.

[m0m0khan]

I am having this issue on JabRef 4.3.1 running on kubuntu. I tried changing my MAC and IP addres both but still getting this error. Can anybody help me?

[AEgit]

@m0m0khan Have you tried using the current developer version of JabRef (https://builds.jabref.org/master/)?

Before using the dev version, create a backup of your JabRef database in case something goes wrong.

[m0m0khan]

I downloaded it, but I am unable to run it (I am new on Linux). This file is tar.gz and I dont know how to excute JabRef after extracting the file. Can you help me here please?

[AEgit]

Uaagg, to be honest, the Linux version is also not running for me at the moment :D
Having said that, though, in my case this is because I want to use the snap version, which is currently blocked due to certain external issues, that are outside of the reach of the JabRef developers.

Coming back to your question: Can you tell me, which distribution of Linux you are running? Is it Ubuntu? Or any other Debian-based distribution? Or anything else?

[m0m0khan]

@AEgit I am using Kubuntu (KDE). Not able to run the dev version yet :(

[AEgit]

Which version of Kubuntu?

[m0m0khan]

@AEgit It's 18.04.3 LTS

[AEgit]

Sorry for taking so long to reply - I had to setup a Linux virtual machine, since I am mainly using Windows when at work. As far as I can tell, the tar file should work immediately, without any issues.
Just untar the file and doubleclick on the JabRef application (found in JabRef-portable_linux/JabRef/bin). This should start JabRef.

If it does not work, you might have to set the execution bit, using chmod +x JabRef in the terminal (assuming that you have moved to the directory, that contains the JabRef file).

Alternatively, the .deb package should work as well, but I have not tested that.

[m0m0khan]

Thanks, It's working!

[AEgit]

Cheers!