Skip to content

Commit

Permalink
Prepend non-HTML links when downloading recursively
Browse files Browse the repository at this point in the history
because its more likely to download temporary links before they expire because its more similar to the browsing experience
  • Loading branch information
john-peterson committed Jan 6, 2015
1 parent 103cbf1 commit dab4254
Showing 1 changed file with 14 additions and 3 deletions.
17 changes: 14 additions & 3 deletions src/recur.c
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,23 @@ url_enqueue (struct url_queue *queue, struct iri *i,
DEBUGP (("[IRI Enqueuing %s with %s\n", quote_n (0, url),
i->uri_encoding ? quote_n (1, i->uri_encoding) : "None"));

if (queue->tail)
queue->tail->next = qel;
queue->tail = qel;
if (html_allowed)
{
if (queue->tail)
queue->tail->next = qel;
queue->tail = qel;
}
else
{
if (queue->head)
qel->next = queue->head;
queue->head = qel;
}

if (!queue->head)
queue->head = queue->tail;
if (!queue->tail)
queue->tail = queue->head;
}

/* Take a URL out of the queue. Return true if this operation
Expand Down

0 comments on commit dab4254

Please sign in to comment.