Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create project from URL #100

Closed
pombredanne opened this issue Feb 13, 2021 · 2 comments
Closed

Create project from URL #100

pombredanne opened this issue Feb 13, 2021 · 2 comments
Assignees
@tdruez
Milestone
1.1

Comments

@pombredanne
Copy link
Contributor

For now we can only provide input as uploaded files or manually add them in the project directory using a shell.
Instead I would like to be able to pass a URL that would be fetched with either FetchCode or the container-inspector fetch module or some other download mechanism (when you may require auth) TBD.
@tdruez
Copy link
Contributor

tdruez commented Feb 15, 2021

Related to #35
@pombredanne can you point me to a working release of FetchCode?

@tdruez tdruez added this to the 1.1 milestone Feb 17, 2021
@tdruez tdruez self-assigned this Feb 17, 2021
tdruez added a commit that referenced this issue Feb 18, 2021
@tdruez
Add a Download URLs field on the project form #100
1ec0e12 
Each URL is fetched and added to the project input directory

Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Feb 25, 2021
@tdruez
Add an input_sources field on the Project model to track inputs origin …
9541980 
…#100

Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Feb 25, 2021
@tdruez
Implement the --input-url option in the management commands #100
f181531 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 1, 2021
@tdruez
Consolidate the downloading logic in a new pipes.fetch module #100
1020b76 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 1, 2021
@tdruez
Refine the upload and download API on the project model #100
2f47379 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 2, 2021
@tdruez
Use a dict format for input_sources in the API #100
a628371 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 2, 2021
@tdruez
Display the source in the project inputs section #100
c1486c1 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 3, 2021
@tdruez
Refine the input source display in the project inputs section #100
e2cc6fb 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 3, 2021
@tdruez
Do not allow input addition once pipelines started to run on project #…
ecf8a5b 
…100

Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 3, 2021
@tdruez
Implement the AddInputsForm and modal #100
ebb34a7 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 3, 2021
@tdruez
Update the documentation with file input related changes #100
c0caa86 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 4, 2021
@tdruez
Update pipeline layout in the project details view to match the list …
3cdc88a 
…view #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 4, 2021
@tdruez
Refine #inputs-panel CSS #100
231271d 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 4, 2021
@tdruez
Refactor forms to remove code duplication #100
8c6729e 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 12, 2021
@tdruez
Add full test coverage for the project input management features #100
d1c1489 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 12, 2021
@tdruez
Refine the add input and add pipeline form modals #100
d872d5e 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 12, 2021
@tdruez
Sort results and expected before assertion in unit test #100
767268f 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Mar 15, 2021
@tdruez
Add ability to add project inputs from URLs (#111)
cd838fe 
* Add a Download URLs field on the project form #100

Each URL is fetched and added to the project input directory

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Add an input_sources field on the Project model to track inputs origin #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Implement the --input-url option in the management commands #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Consolidate the downloading logic in a new pipes.fetch module #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Refine the upload and download API on the project model #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Use a dict format for input_sources in the API #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Display the source in the project inputs section #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Refine the input source display in the project inputs section #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Do not allow input addition once pipelines started to run on project #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Implement the AddInputsForm and modal #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Update the documentation with file input related changes #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Update pipeline layout in the project details view to match the list view #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Refine #inputs-panel CSS #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Refactor forms to remove code duplication #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Add full test coverage for the project input management features #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Refine the add input and add pipeline form modals #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>

* Sort results and expected before assertion in unit test #100

Signed-off-by: Thomas Druez <tdruez@nexb.com>
@tdruez
Copy link
Contributor

tdruez commented Mar 15, 2021

Input files can now be added to projects providing URLs:

  1. Command line: –input-url see https://scancodeio.readthedocs.io/en/latest/scanpipe-command-line.html#scanpipe-add-input-project-project-input-file-files-input-url-urls

  2. Create project form:

Screenshot 2021-03-15 at 8 54 34 AM

  1. REST API:

Screenshot 2021-03-15 at 8 56 28 AM

@tdruez tdruez closed this as completed Mar 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tdruez tdruez

Labels
None yet
Projects
None yet
Milestone
1.1
Development

No branches or pull requests
2 participants
@tdruez @pombredanne