Skip to content

Commit

Permalink
Prepend non-HTML links when downloading recursively
Browse files Browse the repository at this point in the history
because it's more likely to download temporary links before they expire because it's more similar to the browsing experience
  • Loading branch information
john-peterson committed Jan 6, 2015
1 parent 103cbf1 commit ff0ccd6
Showing 1 changed file with 14 additions and 3 deletions.
17 changes: 14 additions & 3 deletions src/recur.c
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,23 @@ url_enqueue (struct url_queue *queue, struct iri *i,
DEBUGP (("[IRI Enqueuing %s with %s\n", quote_n (0, url),
i->uri_encoding ? quote_n (1, i->uri_encoding) : "None"));

if (queue->tail)
queue->tail->next = qel;
queue->tail = qel;
if (html_allowed)
{
if (queue->tail)
queue->tail->next = qel;
queue->tail = qel;
}
else
{
if (queue->head)
qel->next = queue->head;
queue->head = qel;
}

if (!queue->head)
queue->head = queue->tail;
if (!queue->tail)
queue->tail = queue->head;
}

/* Take a URL out of the queue. Return true if this operation
Expand Down

0 comments on commit ff0ccd6

Please sign in to comment.