Add script to create thumbnails/previews of selected images

[mfg92]

This script adds a button 'create thumbnails' to 'selected image[s]' module of darktable's lighttable view. When this button is pressed full sized previews of all selected images are created.

Btw.: I noticed a discrepancy between the lua API description and the implementation of dt_lua_image_t.generate_cache.
In the API documentation the last two parameters are called start_size and end_size.
In the code they are called min and max.
But not only their names are different but also the behaviour is not like described in the API documentation. When max/start_size is greater than min/end_size no image is added to the cache. But the API states "When generating multiple sizes put the largest size as the start size and the smallest as the end".

[wpferguson]

You are correct about the documentation. IIRC I wrote the documentation while the code was still a work in progress. I finally set the code up to use min/max and handle it correctly internally. But, it seems I forgot to go back and update the documentation.

[wpferguson]

Documentation is updated to reflect the way the code works.

EDIT: At least at darktable-org.github.io/luadocs

[wpferguson]

After looking through the code and reading everything, I'm somewhat confused.

Are you trying to generate a full set of cache images, a full resolution preview, or a full size preview, or something else that I didn't name?

Generating caches for sizes 0-8 is a waste of time and a waste of disk space. You only need 3 sizes of cache images (or 2 if you don't use the filmstrip). The sizes are:

• 0 for the filmstrip
• full size preview - 4 if your screen width is 1920 (HD), 5 if your screen width is 2560, etc...
• lighttable thumbnail size - the width of your screen / number of images in a row. Match that to the size chart in the docs. I have a 1920, and a 2560 and the thumbnail cache is size 2 (5 images per row).

Size 8 is the full resolution preview. To produce it requires the pixelpipe and therefore takes a second or 2 on an unprocessed image. I don't bother pre-generating them because the cost is not worth the time and storage. If I'm doing a full preview culling and want to see the full resolution image I just generate it then, with a middle click on the image.

[mfg92]

@wpferguson Ty for your response and correcting the API documentation.

The main purpose of this script for me is to generate the full size 100% previews. Why I need this? I shoot mostly wildlife and macro where you can never be sure if the focus is on the right spot or if there is motion blur in the image. That's why I shoot bursts, so I end up with many images with more or less the same content. Back at home I only want to keep one of the images of a burst. To be able to decide which image to keep I use the preview at 100% (middle mouse button). Generating this image (45MP) takes 3 sec on my PC. When I'm comming home from a shooting with 1000 images is is quite annoying to wait a thousand time 3 seconds.

Maybe I should not generate all thumbnail sizes < 8. But I can not hardcode the sizes which are needed as this script shall be general purpose and not tailored to my system. Perhaps it is possible to determine the width of the darktable window and the amount of rows, then the script could compute which thumbnail sizes are needed.

Another topic: Is it possible to get the information which modifier key was hold when the 'create thumbnails' button was pressed? This would allow me to implement another behaviour in those cases, like only generate the 100% previews when SHIFT is holed or delete all cached images if CTRL is holed.

[ChristianBirzer]

Hi, and thanks a lot for doing my work 🤣.
This is exactly the use case I had in mind when starting with almost the same lua script (I mentioned it in issue #381).

Some ideas and suggestions:

• Instead of only the the print_log I added print_toast when generation was finished or aborted.
• I added an additional button to delete the thumbnails (cached images) with the image:drop_cache(). The main purpose for that was to make testing easier because I can delete and create the previews by a click an see how everything works, but it may be a good idea to have this button to explicitly clear the cache to free disc space. (you suggested deleting with a key modifier, but I'm sure, I won't remember it when I need it -> at least mention it in the buttons tool tip then for me 😊 )
• One thing I thought about but did not implement yet: Would it make sense to add a couple of checkboxes or a min/max dropdown in the preferences to select the cache sizes that shall be generated. In my case I really want the 100% and the lighttable size to allow fast scrolling the first time and instant 100% zoom to check focus and sharpness for a series of almost same pictures (doing wildlife and macro but with only 36MP 🙄). Other sized don't matter for me that much but I don't care about the disc space either.

[mfg92]

@ChristianBirzer: I have implemented the first and scond point of your list of suggestions. I have to think a bit more about the third point.

[mfg92]

The correct tooltip text of the script's buttons do not appear, instead it is always 'images', it looks like a bug to me:

The corresponding documentation can be found here: https://darktable-org.github.io/luadocs/lua.api.manual/darktable/gui/libs/image/#darktableguilibsimageregister_action

@wpferguson Can you have a look at this?

[ChristianBirzer]

I have implemented the first and scond point of your list of suggestions. I have to think a bit more about the third point.

Thanks a lot. A few days ago I played a bit with the preferences (just to learn...) and also thought about my third suggestion.
I think, a couple of checkboxes is not that nice. In the preferences dialog, it won't look very nice when adding 8 checkboxes and the script would have to create each size with a call for its own.
It's easier to have a min/max setting in the prefs but I don't know if this helps that much (maybe just to omit the largest previews to save disc space...?)
If you want to add the min/max setting (with enum type/combo box), you might have a look at darktable-org/darktable#11175

[wpferguson]

@mfg92 darktable-org/darktable#11216

[AlicVB]

Sorry to chime in lately here, but maybe this points can help (or not... in this case just ignore my comment !) :

• it's not mandatory to create thumbnails for all size , as small thumb may be created directly by reducing the size of larger one internally (and it needs very few resources if the sizes are not too different, esp. for small sizes) That can be taken in account to mitigate the disk cache size (and reduce the thumb creation time)
• Saying this, I really thing that a "min-max" widget is needed. Maybe we even can have a simple entry with a syntax like what exists in print dialog : (1;2;5 or 1-7 ...) it will need some extra code to decode the values, but it may be worth the pain...
• I wonder if a specific lib shouldn't be better in this case, as we may have 3 lines (min-max + create + delete) for something not used on daily basis... Using a specific lib allow to fold it, in order to gain some precious vertical space.

[ChristianBirzer]

While doing my tests about preferences, I tried to add such a min-max widget to the lua options page. Using enums it could look like this:

I don't like (sorry for that wording 😉) such a entry field with a special syntax. From a developers view this is cool but from a useres view I don't know what to enter here.
If we really need more than min/max maybe an additional lib would be the best because here we can use a couple of check boxes for the specific sizes.

BTW: Is it possible to add more complex widget layouts to the lua options in preferences? Such as tabbed pages or group boxes? This could make the lua options better readable. (I guess the answer is no because all pages are simple a list of labels and type dependent widgets 🤔)

[wpferguson]

Here's a snapshot of how I'm doing it

The options are

• generate cache on import - automatically generate all the thumbnails I need as soon as import is complete. The only drawback to this is that no other lua script can run until it finishes.
• other cache sizes to generate - my laptop is my main computer with an HD screen (1920 px). My external monitor is 2560 px wide. So full screen preview size is 4 for my laptop and 5 for my external monitor. I list both sizes, separated by a comma, so that whichever size doesn't that get automatically generated, gets generated specifically.
• generate full size preview for culling - Whether or not to generate the full size preview (not full resolution, so in my case 4,5) cache.
• generate culling cache first - I generate the lighttable thumbnails first by default. If you want to start culling with full size previews immediately after export, then they can be generated first and then the lighttable thumbnails.
• screen width in pixels - you already know what this is. I'm going to change it to an enum.

It addition to the above

• I've turned off filmstrip thumbnail generation in mine because I don't use the filmstrip. If I do turn it on I don't notice any lag in the thumbnails appearing. This is probably because my lighttable thumbnail size is 2, so it just grabs those and down sizes them.
• I also automatically regenerate the lighttable thumbnail after an edit, so that if you move from image to image during an editiing session you don't have to wait for all the thumbnails to regenerate when you return to lighttable view.

I see caching as a "set it and forget it" type of application, so I think putting the settings in the lua options is the best place for them rather than taking up space in the lighttable.

BTW: Is it possible to add more complex widget layouts to the lua options in preferences? Such as tabbed pages or group boxes? This could make the lua options better readable. (I guess the answer is no because all pages are simple a list of labels and type dependent widgets thinking)

It could use some love :)

[mfg92]

Sorry for the long delay, but I am unsure about how to proceed.

About @wpferguson implementation: I like the automatic generation of thumbnails on import. But I think additionally to that a button to trigger this manually is needed as well, for example when a user opens an already imported film roll that has no thumbnails yet (e.g. because the thumbnails were deleted in the past).

Furthermore I think we dont have to give the user the option to specify directly which thumbnail size they want, instead we could given them these three options and determin the corresponding thumbnail size (0-8) by using the display resolution:

• generate thumbnails ☑
  • size: 2? 3?
• generate fullscreen previews ☑
  • size: 4-7, depending on display resolution
• generate fullresolution previews ☑
  • size: 8

[ChristianBirzer]

In my opinion (and my workflow) both is very useful (not to say 'needed'). The automatic generation on import and the manual generation.
My workflow in lightroom is like this: Import images from camera and automatically generate the full set of previews. During this time I do something else anyway so I don't care how long this takes.
This allows me to sort out images very quickly including 100% view after I come back.
Very often I also work on past sessions. Due to the long time, the previews were already deleted, so before I start I'll regenerate them.

I think the preconfigured selection you suggest is very handy. Even if I don't know which sizes shall be in which selection. I'd like either have the checkboxes like you suggest where I can select all or have a drop down selection with an additional entry for 'all sizes'. I'm not sure which one is better.