Duplicate Files in SampleTank?

The forum for petitions, theoretical discussion, gripes, or other off topic discussion.

Moderator: James Steele

Forum rules
The forum for petitions, theoretical discussion, gripes, or other matters outside deemed outside the scope of helping users make optimal use of MOTU hardware and software. Posts in other forums may be moved here at the moderators discretion. No politics or religion!!
Post Reply
User avatar
bayswater
Posts: 11955
Joined: Fri Feb 16, 2007 9:06 pm
Primary DAW OS: MacOS
Location: Vancouver

Duplicate Files in SampleTank?

Post by bayswater »

My backup volume for sample libraries is getting full. I suspected I had some files backed multiple times in different folders, and went about looking for duplicates I can delete.

A couple of duplicate file finders report over 1,000 duplicates in my SampleTank sound file backup. A check on a few of these with the terminal Diff command found they are in fact exact duplicates.

Checking the actual ST3 sample library, not the backup, finds about 9G of duplicates, many of them in the same folder, including a 1.5G piano sample set. These are exactly the same file with two different file names. They are not aliases or symbolic links.

Checking other sample libraries revealed very few duplicate files. Disk space is cheap, but these libraries pile up.

Seems to me, if this is real, it's pretty sloppy on IKM's part.
2018 Mini i7 32G 10.14.6, DP 11.3, Mixbus 9, Logic 10.5, Scarlett 18i8
User avatar
mikehalloran
Posts: 15205
Joined: Sun Jan 25, 2009 5:08 pm
Primary DAW OS: MacOS
Location: Sillie Con Valley

Re: Duplicate Files in SampleTank?

Post by mikehalloran »

Happened to me, too. I couldn't figure out it happened but am pretty sure that I did it. Possibly it occurred during a minor version update and the installer switched directories. I really don't know.

I wound up deleting all ST and Miroslav libraries, then reinstalling the latest build.
DP 11.31; 828mkII FW, micro lite, M4, MTP/AV USB Firmware 2.0.1
2023 Mac Studio M2 8TB, 192GB RAM, OS Sonoma 14.4.1, USB4 8TB external, M-Audio AIR 192|14, Mackie ProFxv3 6/10/12; 2012 MBPs Catalina, Mojave
IK-NI-Izotope-PSP-Garritan-Antares, LogicPro X, Finale 27.4, Dorico 5.2, Notion 6, Overture 5, TwistedWave, DSP-Q 5, SmartScore64 Pro, Toast 20 Pro
User avatar
mhschmieder
Posts: 11283
Joined: Wed Jul 06, 2005 10:01 pm
Primary DAW OS: MacOS
Location: Annandale VA

Re: Duplicate Files in SampleTank?

Post by mhschmieder »

I seem to have to add 1 TB a year at this point, which is ridiculous even that today's prices and presents backup challenges.

I was just thinking about this today, how many vendors add duplicate files and/or do sloppy bookkeeping on updates. I'm usually too scared to delete stuff "just in case", and also as it can be time-consuming to load every Kontakt preset and make sure all samples are still found.

On top of that, there are missing samples in quite a few commercial products, and vendors DO NOT GET BACK TO ME even when I spend a lot of my own time helping them to quickly resolve the issue should they choose to bother. This also lowers my faith in safely deleting apparently excess files.

One area where I think a lot of disc space could be saved is in deleting 16-bit samples (and associated presets) where 24-bit samples are also loaded, and similarly with low sample rate versions of products that have high sample rate versions side by side.

Of course, even that would be time-consuming to cull, so maybe not worth it.

The bigger culprit though, seems to be EXACT duplicates that are due to sloppy installers that either leave behind too many recursive artifacts or don't clean up unneeded older files.
iMac 27" 2017 Quad-Core Intel i5 (3.8 GHz, 64 GB), OSX 13.6.1, MOTU DP 11.31, iZotope RX 10
RME Babyface Pro FS, Radial JDV Mk5, Hammond XK-4, Moog Voyager

Eugenio Upright, 60th Anniversary P-Bass, USA Geddy Lee J-Bass, Yamaha BBP35
Select Strat, 70th Anniversary Esquire, Johhny Marr Jaguar, 57 LP, Danelectro 12
Eastman T486RB, T64/V, Ibanez PM2, D'angelico Deluxe SS Bari, EXL1
Guild Bari, 1512 12-string, M20, Martin OM28VTS, Larivee 0040MH
Post Reply