Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run fetcher tests on a weekly basis #5807

Merged
merged 2 commits into from
Jan 8, 2020
Merged

Run fetcher tests on a weekly basis #5807

merged 2 commits into from
Jan 8, 2020

Conversation

koppor
Copy link
Member

@koppor koppor commented Jan 2, 2020

Sometimes, our fetchers break (see e.g. #5804) or an endpoint being down (e.g., ottobib this night).

  Test searchForIsbnAvailableAtOttoBibButNonOnEbookDe() FAILED (31.4s)

  org.jabref.logic.importer.FetcherException: A network error occurred
      at org.jabref.logic.importer.fetcher.IsbnFetcherTest.searchForIsbnAvailableAtOttoBibButNonOnEbookDe(IsbnFetcherTest.java:104)
  Caused by: java.net.ConnectException: Connection timed out
      at org.jabref.logic.importer.fetcher.IsbnFetcherTest.searchForIsbnAvailableAtOttoBibButNonOnEbookDe(IsbnFetcherTest.java:104)

This proposal triggers fetcher tests a) only of fetchers code changes and b) once a week (to check for general issues such as changed endpoints at the provider's side)

It is accepted that refactorings in the internal data model (where fetchers rely on) are not tested any more.

@koppor koppor added the fetcher label Jan 2, 2020
@koppor koppor added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Jan 2, 2020
@tobiasdiez
Copy link
Member

What's the disadvantage with the current workflow where fetcher tests are always invoked?

@koppor
Copy link
Member Author

koppor commented Jan 2, 2020

Fetchers break now and then. We are not quick at fixing the fetchers. Then, PRs get a red cross even though they are fine. This is hard to explain to beginners.

In case we (as team) agree that we disable fetcher tests as soon as the remote breaks to keep PRs breaking only for one day or so, I would keep everything as is (plus additionally a weekly run)

Copy link
Member

@tobiasdiez tobiasdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I've no strong opinion about this. The important thing is that the fetchers are checked regularly so that we don't end up with 20 broken fetchers before we want to release. Both options (scheduled vs every build) provide this and each have different advantages and disadvantages...

branches:
- master
paths:
- 'src/main/java/org/jabref/logic/importer/fetcher/**'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe its better to be more general and include src/main/java/org/jabref/logic/importer as the parsers are also essential for the fetcher and are a potential source why a fetcher test may break.

@koppor koppor merged commit 36e503b into master Jan 8, 2020
@koppor koppor deleted the schedule-fetcher-tests branch January 8, 2020 06:09
@koppor koppor mentioned this pull request Aug 2, 2020
@koppor koppor removed the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Aug 2, 2020
koppor pushed a commit that referenced this pull request Jan 1, 2022
f78c707 Update Turabian (full note) for 17.1.3.2 Reprint Editions (#5809)
2169ddc Create art-libraries-society-of-north-america-arlisna-reviews.csl (#5791)
94119a6 Create university-of-south-wales-harvard.csl (#5787)
470925c Update fachhochschule-sudwestfalen.csl (#5811)
1a9eff7 Create annals-of-public-and-cooperative-economics.csl (#5747)
d7477f2 Merge pull request #5368 from POBrien333/patch-862
113e683 Merge pull request #5812 from citation-style-language/apa-no-initials
62dacff Create universitat-oberta-de-catalunya-apa.csl
046482c Fix Organization Studies title
1213b95 Make oranization-studies dependent
c9641f4 Create apa-no-initials.csl
0b4e047 Modify american-journal-of-archaeology.csl (#5719)
ddeda4f Create journal-of-economic-impact.csl (#5707)
40c2696 Update gallia.csl (#5782)
5d5927a Create serbian-archives-of-medicine.csl (#5721)
2777954 Merge pull request #5807 from dhacker29/master
77e3fca Update Society of Biblical Literature Full Note for 6.1.6
4479a5a haaga-helia-university-of-applied-sciences-harvard.csl: no date cites need to be unique (#5726)
d00e046 Create university-of-roehampton-harvard.csl (#5732)
e76da40 Medicinski Razgledi -- fix locale
75b1554 Add original-date extra to proper place as per SBL Handbook 6.2.17-18 (#5725)
d43676e Update harvard-bournemouth-university.csl (#5667)
0312a39 Create ABNT NBR 6023:2018. UNESP – Faculdade de Engenharia de Guarati… (#5705)
6585ff9 Create sinergie-italian-journal-of-management.csl (#5776)
a7dc678 Create journal-of-global-health.csl (#5775)
02080ce Merge pull request #5800 from citation-style-language/validation-fixes
8b51d85 change s. d. to s.d. in french style for INBO reports (issue #2) (#5634)
1b4ec09 Create bloomsbury-academic.csl (#5741)
1f28d87 Create zeithistorische-forschungen.csl (#5766)
bb11942 Remove superfluous 'container' data from wiley-vch-books
06038a7 Remove et-al affixes
be96bd8 Remove et-al affixes from bibliotecae-it.csl
03a94e4 Remove more et-al affixes
07d7423 Revert non-et-al changes to universitat-basel-iberomanistik
f8e5fd8 Create medicinski-razgledi.csl (#5789)
b4df8ee Create dut-harvard.csl (#5762)
40855c4 Fix affixes on et-al
b4fe9b0 Merge pull request #5798 from citation-style-language/uris
3dc4aaf URL-encode documentation URLs
408a3a8 fixes citation-style-language/Sheldon#38
42aca5b Update biophysics-and-physicobiology.csl (#5751)
706bea0 Create universidade-do-porto-faculdade-de-engenharia-chicago-pt.csl (#5742)
269ff91 Update universidade-do-porto-faculdade-de-engenharia-chicago.csl (#5797)
0ba51e7 Update el-profesional-de-la-informacion.csl (#5785)
dac3baf Create arthropod-systematics-and-phylogeny.csl (#5772)
9dff241 Merge pull request #5796 from dhacker29/patch-1
35949f4 Update turabian-fullnote-bibliography-no-ibid.csl
ccfb661 Merge pull request #5793 from benjaminmoon/patch-2
520e127 Update frontiers-medical-journals.csl (#5740)
ab629ab Update natura-croatica.csl (#5795)
60b2b17 Rename turabian-fullnote-bibliography-no-ibid to turabian-fullnote-bibliography-no-ibid.csl
ad4edff Create turabian-fullnote-bibliography-no-ibid
d9b956c Update monographs-of-the-palaeontographical-society.csl
3648ae6 Merge pull request #5792 from StianOby/fix-no-page-number-articles
e15b2f7 Update norsk-henvisningsstandard-for-rettsvitenskapelige-tekster.csl
d8778cf Removed "page" (s. ) term for articles that do not have page numbers.
409017c add medium field for eBooks to MLA 8th

git-subtree-dir: buildres/csl/csl-styles
git-subtree-split: f78c707
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants