Media Library refresh should be more clever

[jim2005]

Hello,
I think this is a bug, not a future request.

In Winamp 5.x (as well as in the previous WACUP preview, or so it seemed to me), refreshing the Local Library was very quick. Somehow the Media Library plugin was able to detect the changes in the metadata or the file structure of the library and scan only those files that had really changed.

Since the build #17040 (I guess this is the first time ml_ll plugin was introduced in public builds, pls correct if I'm wrong), when I perform a "Refresh metadata for all library item(s)"), the whole library structure is scanned as if it was the first time. Result is that the time to complete this action is very long (my library consists of 250,000 songs) and it's always this long even when the changes are very small (e.g. adding a file or changing ID3 metadata in a dozen of songs). I pruned and compacted the library but (unsurprisingly) it didn't improve.

Am I missing something? Is it a feature to be implement on a future build?
Is it possible to temporarily switch to the legacy (ml_local ?) plugin (and where can I obtain it?) in order to check if it worked better?

Thank you

[Aminifu]

The recent beta releases have a feature in the preferences media library group called folder monitor that does what you're asking for. I don't know if it is also in the current public release. Look for it.

[jim2005]

The recent beta releases have a feature in the preferences media library group called folder monitor that does what you're asking for. I don't know if it is also in the current public release. Look for it.

I checked it and it, happily, exists in the preview build too. It was disabled in my installation. Is it (i.e. disabled) the default setting? If yes, I think it should be changed since it's an easily overlooked option.

Now that I have enable it, WACUp detects folder structure and metadata changes (like lyrics).
Many thanks!

[Aminifu]

Yeah, great!  IIRC, that feature did have a few issues when it was first introduced and I think it is disabled by default, which I had forgotten.

WACUP does have a lot of options. I force myself to look at all the preferences with every new release. Every now and then something useful pops up. It's also a good idea to take the time to carefully read the changelogs.

[dro]

I think I see what the some of the confusion is now I've had a look back at what the ml_local options were.

The ml_ll refresh option is intentionally set to re-read the files because the modified file time can't always be relied upon as there's tools that don't always modify it whereas ml_local offers a differently named option that is a trigger option for its watch folders feature which afaict would just look for overall file differences (time changed / not present).

I will look to add an equivalent folder monitor trigger to ml_ll's library button sub-menu though I'm likely going to do it so specific folder monitor entries can be triggered for those that are enabled at the time or something along those lines (will see what happens when I start coding things).


The ml_ll folder monitor is equivalent to ml_local's watch folder handling & it should be importing the folder list from ml_local's settings though the comment about it being enabled by default, I'm not sure what's meant by that. As the prefs is always going to be there & the logic is if it's present in the imported settings then any folders should be auto-enabled on an ~6hr default interval between scans vs any other config options such as how it's done on initial loading (as unlike ml_local I try to avoid it spinning up too soon if doing quick load & close / post-crash issues).

There's even an option on the import folder dialog which allows for them to be added to the folder monitor but that isn't enabled by default because I've seen too many cases of sub-folders being added which'd then cause the folder monitor to be looking at many folders when it's not appropriate - maybe that's what is being referred to as not enabled by default ?


One final thing, ml_ll's metadata reading is more involved than ml_local's since it collects more information about the files such as format information & associated albumart which obviously can take longer to determine per file despite doing what is reasonable to try to minimise how long it overall takes to do all of that.

So even if it only takes a few milliseconds per file (which isn't bad), that over 250k of files is going to easily be in the 12-15min range & I have been debating making ml_ll look at 2 files at a time instead of just 1 which could reduce the overall time but it could be slower depending on the drive contention & how well the input plug-ins deal with multi-threaded access (there's some that don't like it at all).

-dro

[Aminifu]

...

So even if it only takes a few milliseconds per file (which isn't bad), that over 250k of files is going to easily be in the 12-15min range & I have been debating making ml_ll look at 2 files at a time instead of just 1 which could reduce the overall time but it could be slower depending on the drive contention & how well the input plug-ins deal with multi-threaded access (there's some that don't like it at all).

-dro

Just my 2 cents. It seems that trying to look at 2 files at a time is not worth the effort. The folder monitor is great for quickly picking up a few changes in a large database. The in-dept refresh is also useful at times, even if/when it takes awhile to finish.

[dro]

Without trying it out I'm not sure but it's something I've got on my of things to consider once I can be certain of the core input plug-ins being able to cope with it. The folder monitor handling is itself multi-threaded so it checks for files to consider & then hands that over to a different thread which then does the metadata reading allowing for the initial scan to generally take less time. That's still hitting multiple files on the drive at the same time due to the overlapping of things relying somewhat on the OS / file system cache to minimise the effect of it.

-dro