-
-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
* Adds feature CleanupJob for removing non-existent files (#2) * feat: RemoveLinksToNotExistentFiles.java * feat: added option to cleanup entries * feat: improved cleanup entry dialogue * Test: implements test for linked files which exist Implementation of a test which calls the cleanup functionality of RemoveLinksToNotExistentFiles with a entry that is linked to an existing file. The test makes sure that no changes are made since the linked file exists. * Refactor: refactored test to match the standard of the project Removed the JavaDoc comment from the test, since other tests in the project don't have JavaDoc:s. Also removed test from the name of the test function, as it seems that the standard way of naming tests is to not have test in the testfunctions name. * Refactor: removed system.out.print() Removed an print from the tests. * fix: non english character deleted * Docs: Added information about changes Added information about the changes made for solving issue#10929 to the CHANGES.md file. --------- * Fix: Removes unused imports and solves other errors Removes unused imports and solves the error of language keys that were not in the english laguage file. * Fix: Removes empty line and solves other issues Removes an empty line where 2 empty lines appear in after each other. Also adds brackets to URL and removes an key which is not in the english language file and was missed in the last commit. * Docs: Removes technicality from changelog Removes technical details for the change made in changelog and also follows the suggestion for removing the space and brackets between the link and the issue. * test: enhanced RemoveLinksToNotExistentFilesTest (#3) * test: enhanced RemoveLinksToNotExistentFilesTest * test: add real content * test: change structure * test: added expectedChanges * test: added a third test to RemoveLinksToNotExistentFilesTest * refactor: removed unnecessary toString * style: removed a whitespace * refactor: checkstyle formatting. formatted files according to checkstyle plugin. --------- * Resolve request changes (#4) * test: enhanced RemoveLinksToNotExistentFilesTest * test: add real content * test: change structure * test: added expectedChanges * test: added a third test to RemoveLinksToNotExistentFilesTest * refactor: removed unnecessary toString * style: removed a whitespace * refactor: checkstyle formatting. formatted files according to checkstyle plugin. * refactor: replaced .get(0) with .getFirst() for first accessing element of List. Changed was required from ./gradlew rewriteDryRun --------- * Resolve more request changes and warnings (#5) * Reverse the condition and the content in RemoveLinksToNotExistentFiles Resolves comment "Reverse the condition and the content. Reason: You have BOTH (true and false) cases covered. The true case should normally come first." * Fix: Store bibFolder.resolve("test.bib") in a variable * Refactor: Rename variable fileFolder to originalFileFolder and remove comment. * Refactor: Rename defaultFileFolder to newFileFolder * Refactor: Fix indentation in RemoveLinksToNotExistentFilesTest * Refactor: Replaced Arrays.asList() with List.of() * Refactor: Replaced Arrays.asList() with List.of() * Refactor: Change to use java.nio for file deletion. * Refactor: Move comment to line above. * Refactor: Added "PDF" as third argument to LinkedFile for OnlineLink files * Refactor: Removed unused variables in RemoveLinksToNotExistentFiles * Refactor: Throw IOException in RemoveLinksToNotExistentFilesTest functions * Refactor: Tests assert using List.of() in RemoveLinksToNotExistentFilesTest * Refactor: Corrected Arguments for LinkedFile in RemoveLinksToNotExistentFilesTest * Refactor: Class fields into local variables This refactor fixed 3 warnings. * docs: updated docs to remove merge conflict --------- Co-authored-by: karlsb <36365664+karlsb@users.noreply.github.com> Co-authored-by: burcukilic <94201593+burcukilic@users.noreply.github.com>
- Loading branch information
3 people
committed
Mar 6, 2024
1 parent
f5efb34
commit 985b40b
Showing
7 changed files
with
257 additions
and
19 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
52 changes: 52 additions & 0 deletions
52
src/main/java/org/jabref/logic/cleanup/RemoveLinksToNotExistentFiles.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| package org.jabref.logic.cleanup; | ||
|
|
||
| import java.nio.file.Path; | ||
| import java.util.ArrayList; | ||
| import java.util.Collections; | ||
| import java.util.List; | ||
| import java.util.Objects; | ||
| import java.util.Optional; | ||
|
|
||
| import org.jabref.model.FieldChange; | ||
| import org.jabref.model.database.BibDatabaseContext; | ||
| import org.jabref.model.entry.BibEntry; | ||
| import org.jabref.model.entry.LinkedFile; | ||
| import org.jabref.model.util.OptionalUtil; | ||
| import org.jabref.preferences.FilePreferences; | ||
|
|
||
| public class RemoveLinksToNotExistentFiles implements CleanupJob { | ||
| private final BibDatabaseContext databaseContext; | ||
| private final FilePreferences filePreferences; | ||
|
|
||
| public RemoveLinksToNotExistentFiles(BibDatabaseContext databaseContext, FilePreferences filePreferences) { | ||
| this.databaseContext = Objects.requireNonNull(databaseContext); | ||
| this.filePreferences = Objects.requireNonNull(filePreferences); | ||
| } | ||
|
|
||
| @Override | ||
| public List<FieldChange> cleanup(BibEntry entry) { | ||
| List<LinkedFile> files = entry.getFiles(); | ||
| List<LinkedFile> cleanedUpFiles = new ArrayList<>(); | ||
| boolean changed = false; | ||
| for (LinkedFile file : files) { | ||
| if (file.isOnlineLink()) { | ||
| cleanedUpFiles.add(file); | ||
| } else { | ||
| Optional<Path> oldFile = file.findIn(databaseContext, filePreferences); | ||
|
|
||
| if (oldFile.isEmpty()) { | ||
| changed = true; | ||
| } else { | ||
| cleanedUpFiles.add(file); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (changed) { | ||
| Optional<FieldChange> changes = entry.setFiles(cleanedUpFiles); | ||
| return OptionalUtil.toList(changes); | ||
| } | ||
|
|
||
| return Collections.emptyList(); | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
163 changes: 163 additions & 0 deletions
163
src/test/java/org/jabref/logic/cleanup/RemoveLinksToNotExistentFilesTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,163 @@ | ||
| package org.jabref.logic.cleanup; | ||
|
|
||
| import java.io.IOException; | ||
| import java.nio.file.Files; | ||
| import java.nio.file.Path; | ||
| import java.util.List; | ||
|
|
||
| import org.jabref.logic.bibtex.FileFieldWriter; | ||
| import org.jabref.model.FieldChange; | ||
| import org.jabref.model.database.BibDatabase; | ||
| import org.jabref.model.database.BibDatabaseContext; | ||
| import org.jabref.model.entry.BibEntry; | ||
| import org.jabref.model.entry.LinkedFile; | ||
| import org.jabref.model.entry.field.StandardField; | ||
| import org.jabref.model.entry.types.StandardEntryType; | ||
| import org.jabref.model.metadata.MetaData; | ||
| import org.jabref.preferences.FilePreferences; | ||
|
|
||
| import org.junit.jupiter.api.BeforeEach; | ||
| import org.junit.jupiter.api.Test; | ||
| import org.junit.jupiter.api.io.TempDir; | ||
|
|
||
| import static org.junit.jupiter.api.Assertions.assertEquals; | ||
| import static org.mockito.Mockito.mock; | ||
| import static org.mockito.Mockito.when; | ||
|
|
||
| public class RemoveLinksToNotExistentFilesTest { | ||
| private Path fileBefore; | ||
| private BibEntry entry; | ||
| private RemoveLinksToNotExistentFiles removeLinks; | ||
|
|
||
| @BeforeEach | ||
| void setUp(@TempDir Path bibFolder) throws IOException { | ||
| // The folder where the files should be moved to | ||
| Path newFileFolder = bibFolder.resolve("pdf"); | ||
| Files.createDirectory(newFileFolder); | ||
|
|
||
| Path originalFileFolder = bibFolder.resolve("files"); | ||
| Path testBibFolder = bibFolder.resolve("test.bib"); | ||
| Files.createDirectory(originalFileFolder); | ||
| fileBefore = originalFileFolder.resolve("test.pdf"); | ||
| Files.createFile(fileBefore); | ||
|
|
||
| MetaData metaData = new MetaData(); | ||
| metaData.setDefaultFileDirectory(newFileFolder.toAbsolutePath().toString()); | ||
|
|
||
| BibDatabaseContext databaseContext = new BibDatabaseContext(new BibDatabase(), metaData); | ||
| Files.createFile(testBibFolder); | ||
| databaseContext.setDatabasePath(testBibFolder); | ||
|
|
||
| LinkedFile fileField = new LinkedFile("", fileBefore.toAbsolutePath(), ""); | ||
|
|
||
| // Entry with one online and one normal linked file | ||
| entry = new BibEntry(StandardEntryType.Article) | ||
| .withField(StandardField.AUTHOR, "Shatakshi Sharma and Bhim Singh and Sukumar Mishra") | ||
| .withField(StandardField.DATE, "April 2020") | ||
| .withField(StandardField.YEAR, "2020") | ||
| .withField(StandardField.DOI, "10.1109/TII.2019.2935531") | ||
| .withField(StandardField.FILE, FileFieldWriter.getStringRepresentation(List.of( | ||
| new LinkedFile("", "https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8801912", "PDF"), | ||
| fileField))) | ||
| .withField(StandardField.ISSUE, "4") | ||
| .withField(StandardField.ISSN, "1941-0050") | ||
| .withField(StandardField.JOURNALTITLE, "IEEE Transactions on Industrial Informatics") | ||
| .withField(StandardField.PAGES, "2346--2356") | ||
| .withField(StandardField.PUBLISHER, "IEEE") | ||
| .withField(StandardField.TITLE, "Economic Operation and Quality Control in PV-BES-DG-Based Autonomous System") | ||
| .withField(StandardField.VOLUME, "16") | ||
| .withField(StandardField.KEYWORDS, "Batteries, Generators, Economics, Power quality, State of charge, Harmonic analysis, Control systems, Battery, diesel generator (DG), distributed generation, power quality, photovoltaic (PV), voltage source converter (VSC)"); | ||
|
|
||
| FilePreferences filePreferences = mock(FilePreferences.class); | ||
| when(filePreferences.shouldStoreFilesRelativeToBibFile()).thenReturn(false); | ||
| removeLinks = new RemoveLinksToNotExistentFiles(databaseContext, filePreferences); | ||
| } | ||
|
|
||
| @Test | ||
| void deleteFileInEntryWithMultipleFileLinks() throws IOException { | ||
| LinkedFile fileField = new LinkedFile("", fileBefore.toAbsolutePath(), ""); | ||
| FieldChange expectedChange = new FieldChange(entry, StandardField.FILE, | ||
| FileFieldWriter.getStringRepresentation(List.of( | ||
| new LinkedFile("", "https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8801912", "PDF"), | ||
| fileField)), | ||
| FileFieldWriter.getStringRepresentation(new LinkedFile("", "https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8801912", "PDF")) | ||
| ); | ||
| BibEntry expectedEntry = new BibEntry(StandardEntryType.Article) | ||
| .withField(StandardField.AUTHOR, "Shatakshi Sharma and Bhim Singh and Sukumar Mishra") | ||
| .withField(StandardField.DATE, "April 2020") | ||
| .withField(StandardField.YEAR, "2020") | ||
| .withField(StandardField.DOI, "10.1109/TII.2019.2935531") | ||
| .withField(StandardField.FILE, FileFieldWriter.getStringRepresentation( | ||
| new LinkedFile("", "https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8801912", "PDF"))) | ||
| .withField(StandardField.ISSUE, "4") | ||
| .withField(StandardField.ISSN, "1941-0050") | ||
| .withField(StandardField.JOURNALTITLE, "IEEE Transactions on Industrial Informatics") | ||
| .withField(StandardField.PAGES, "2346--2356") | ||
| .withField(StandardField.PUBLISHER, "IEEE") | ||
| .withField(StandardField.TITLE, "Economic Operation and Quality Control in PV-BES-DG-Based Autonomous System") | ||
| .withField(StandardField.VOLUME, "16") | ||
| .withField(StandardField.KEYWORDS, "Batteries, Generators, Economics, Power quality, State of charge, Harmonic analysis, Control systems, Battery, diesel generator (DG), distributed generation, power quality, photovoltaic (PV), voltage source converter (VSC)"); | ||
|
|
||
| Files.delete(fileBefore); | ||
| List<FieldChange> changes = removeLinks.cleanup(entry); | ||
|
|
||
| assertEquals(List.of(expectedChange), changes); | ||
| assertEquals(expectedEntry, entry); | ||
| } | ||
|
|
||
| @Test | ||
| void keepLinksToExistingFiles() { | ||
| LinkedFile fileField = new LinkedFile("", fileBefore.toAbsolutePath(), ""); | ||
| BibEntry expectedEntry = new BibEntry(StandardEntryType.Article) | ||
| .withField(StandardField.AUTHOR, "Shatakshi Sharma and Bhim Singh and Sukumar Mishra") | ||
| .withField(StandardField.DATE, "April 2020") | ||
| .withField(StandardField.YEAR, "2020") | ||
| .withField(StandardField.DOI, "10.1109/TII.2019.2935531") | ||
| .withField(StandardField.FILE, FileFieldWriter.getStringRepresentation(List.of( | ||
| new LinkedFile("", "https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8801912", "PDF"), | ||
| fileField))) | ||
| .withField(StandardField.ISSUE, "4") | ||
| .withField(StandardField.ISSN, "1941-0050") | ||
| .withField(StandardField.JOURNALTITLE, "IEEE Transactions on Industrial Informatics") | ||
| .withField(StandardField.PAGES, "2346--2356") | ||
| .withField(StandardField.PUBLISHER, "IEEE") | ||
| .withField(StandardField.TITLE, "Economic Operation and Quality Control in PV-BES-DG-Based Autonomous System") | ||
| .withField(StandardField.VOLUME, "16") | ||
| .withField(StandardField.KEYWORDS, "Batteries, Generators, Economics, Power quality, State of charge, Harmonic analysis, Control systems, Battery, diesel generator (DG), distributed generation, power quality, photovoltaic (PV), voltage source converter (VSC)"); | ||
|
|
||
| List<FieldChange> changes = removeLinks.cleanup(entry); | ||
|
|
||
| assertEquals(List.of(), changes); | ||
| assertEquals(expectedEntry, entry); | ||
| } | ||
|
|
||
| @Test | ||
| void deleteLinkedFile() throws IOException { | ||
| LinkedFile fileField = new LinkedFile("", fileBefore.toAbsolutePath(), ""); | ||
|
|
||
| // There is only one linked file in entry | ||
| entry.setField(StandardField.FILE, FileFieldWriter.getStringRepresentation(fileField)); | ||
| FieldChange expectedChange = new FieldChange(entry, StandardField.FILE, | ||
| FileFieldWriter.getStringRepresentation(fileField), | ||
| null); | ||
| BibEntry expectedEntry = new BibEntry(StandardEntryType.Article) | ||
| .withField(StandardField.AUTHOR, "Shatakshi Sharma and Bhim Singh and Sukumar Mishra") | ||
| .withField(StandardField.DATE, "April 2020") | ||
| .withField(StandardField.YEAR, "2020") | ||
| .withField(StandardField.DOI, "10.1109/TII.2019.2935531") | ||
| .withField(StandardField.ISSUE, "4") | ||
| .withField(StandardField.ISSN, "1941-0050") | ||
| .withField(StandardField.JOURNALTITLE, "IEEE Transactions on Industrial Informatics") | ||
| .withField(StandardField.PAGES, "2346--2356") | ||
| .withField(StandardField.PUBLISHER, "IEEE") | ||
| .withField(StandardField.TITLE, "Economic Operation and Quality Control in PV-BES-DG-Based Autonomous System") | ||
| .withField(StandardField.VOLUME, "16") | ||
| .withField(StandardField.KEYWORDS, "Batteries, Generators, Economics, Power quality, State of charge, Harmonic analysis, Control systems, Battery, diesel generator (DG), distributed generation, power quality, photovoltaic (PV), voltage source converter (VSC)"); | ||
|
|
||
| Files.delete(fileBefore); | ||
| List<FieldChange> changes = removeLinks.cleanup(entry); | ||
|
|
||
| assertEquals(List.of(expectedChange), changes); | ||
| assertEquals(expectedEntry, entry); | ||
| } | ||
| } |