Clicky

Hybrid
Hello!

I'm a photographer and I have a lot of RAW files (Nikon NEF specifically). Often, keywords are written to .XMP sidecar files instead of to the RAW file itself. When I search for keywords, spotlight finds them within the XMP files but it has no idea about the original .NEF RAW file that the sidecar XMP file relates to.

Ideally I'd love to be able to search for a keyword in the XMP and have the search result return the RAW file. Only returning the XMP file isn't much use, especially if I want to preview or open the RAW file directly...

Is it possible to somehow link the XMP result to a NEF file? Or get Spotlight to apply keywords from the .XMP file to the associated .NEF file? Does Spotlight need an importer for the XMP data?

This seems like such a core feature but so far I haven't had any luck... Only formats like JPEG/TIFF etc work because the XMP keywords get embedded in the file itself.

Any suggestions or help would be much appreciated!

Thanks,

Stephen
0 0
houdah
Hi!

Unfortunately Mac OS X and Spotlight don't really handle sidecar files.
Spotlight indexes both the image and the sidecar files individually and could report either one in the search results. Since the value you search for is only stored in the sidecar, that will show up in the results list.

I wonder if HoudahSpot could offer an option to locate the image file when you double-click on a XMP sidecar in results.
Wonder what HoudahSpot should offer if more than one image file matches the sidecar.

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
Hybrid
Hi Pierre!

Thank you for your reply! That makes sense and I thought that's how OS X probably handled it... I don't really understand how the metadata importers work and that's why I was wondering if the importer could read an XMP but associate the information with the NEF/RAW file it finds. It's probably a lot more complicated and could be unreliable if someone deletes the RAW file but not the XMP. hmmm...

It would be pretty cool if HoudahSpot could make that connection between the XMP and the RAW file. It should be a relatively straightforward task and I don't think there should ever be more than one matching file - the XMP file is always named the same as the RAW file (eg. DSC_0001.XMP -> DSC_0001.NEF). The filename of the RAW file is also stored in the XMP data.

Of course there could be multiple files with the same name on the system, but within the scope of a directory, there can only be one match. So maybe that could be used to make this work - if you find an XMP file, look within the same directory to see if a RAW file matches and if a match is found, display the RAW file instead of the XMP. This would also make it nice for being able to see thumbnails straight away.

Do you think this sounds possible? It would be a really nice feature and as far as I'm aware no other software offers this (except maybe this: http://www.helios.de/web/EN/news/xmp_indexing.html - but that's a server based solution and seems really over-complicated for this problem).

I'm sure there are plenty of other photographers out there who would find this an invaluable feature :)

Cheers,

Stephen
0 0
houdah
Hi Stephen,

I had not thought about previews. My idea of opening the image file rather than the sidecar is not good enough.

An elegant solution would be to have a system-wide QuickLook plug-in to preview XMP files.
That plug-in would provide the actual image to QuickLook when asked for a preview of the sidecar.

Best,
Pierre Bernard
Houdah Software s.à r.l.

Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
Hybrid
Hi Pierre,

That's a brilliant solution! I really like the idea of a QuickLook plugin and it would mean you don't have to mess with your application. I don't know much about QuickLook plugins but would it provide Finder / HoudahSpot with the image for the thumbnail in icon view / cover flow also? Being able to preview the file in QuickLook would be nice but having it work with thumbnails like real image files would be even better!

For opening the files, there would rarely be a reason you'd want to open the XMP file itself so maybe a small helper application could handle the XMP file and redirect it to open the associated RAW file. If that worked, it should close the loop nicely on the system.

Do you know much about QuickLook plugins or have any suggestions where I might start with finding or developing one that does this?

Really appreciate your help! Thanks!

Stephen
0 0
houdah
Hi Stephen,

I have never written a QuickLook importer. But that should no be hard.

I have searched the web for a XMP QuickLook importer, but found none.

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
Hybrid
Hi Pierre,

I also searched and haven't been able to find an existing QuickLook plugin for XMP files. Is it something you would have an interest in trying to build? I understand if you don't have time or the motivation but I'm sure you'd do a better job than I could!

Cheers,

Stephen
0 0
houdah
Hi Stephen,

I currently have no plans to write an QuickLook plug-in
But I'll keep the idea in mind.

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
Hybrid
Hi Pierre,

Thanks, that would be great :) I'll continue looking for solutions too. I really appreciate all your time and input so far!

Cheers,

Stephen
0 0
robgendreau
Anyone find any solutions to the xmp sidecar issue, or work-arounds? Seems like more and more people use RAW files now, so I would have thought there'd be at least an importer for xmp files. Once you can find the XMP sidecar you can at least manually find the RAW.

Currently I have to do a work-around that essentially uses a jpg pair (I have to shoot RAW+JPG for this to work, but you could generate a jpg pair easily enough) as the "sidecar" for the RAW. Since I can embed the same keywords and other metadata in the jpg, Spotlight finds it. Then I just copy the filename minus suffix, drop it into the "name contains" field, and change the search  to ANY is true. Then it shows the jpg, the RAW, and the XMP sidecar. Since Apple is rather tardy in RAW support for one of my cameras I need the jpgs anyway. But in the long run, a workflow that didn't require the jpgs would be more efficient.

And BTW, is there a way that anyone knows of that once you find an image file in HoudahSpot that you could then find that image within Lightroom? You can "open" the image "in" Lightroom, but since LR is really a database with referenced images it will just offer to import that image, even if it's already in LR. So I'm looking more for a script or Automator workflow that would "find in Lightroom," in essence.
0 0
Hybrid
Hi Rob,

I haven't found a workable solution for this yet but I'm still hopeful it will be possible one day. XMP files will show up in spotlight searches for me and having a simple helper application that looks at the .XMP and redirects you to the associated RAW file would be nice. The ideal is that functionality plus a QuickLook plugin that shows a preview of the image in the place of the XMP. Then the XMP files would behave like images...

I investigated further and found some interesting work here: http://www.capturemonkey.com/quicklook-lightroom - I wrote to the author asking him about the possibility of an XMP plugin that shows the RAW image in the preview / thumbnail and he said he'd already tried that but the QuickLook API didn't allow rendering from a different file to the one you were looking at. He looked into using a separate process for this but there were concerns about sandboxing and Apple not allowing it in the App Store. Using other OS X APIs for rendering the preview is apparently too slow (3-5s according to him) so he gave up on the idea. It might be worth asking again to see if there has been any progress.

I'm still hopeful someone will try because there are other ways around the problem of rendering time, like using the built-in JPG preview from the RAW file instead of trying to render it from scratch.

One other possibility is embedding the keywords directly in your RAW files. Photo Mechanic has the ability to do this and in the past this is what I've done (it is reversible too, restoring your original RAW file) but I have reservations about modifying the RAW files so I'd rather not have to use this approach. It also has the disadvantage of making backups and file verification harder because it modifies the RAW file whenever you change your metadata.

Of course, you could convert all your files to DNG and this solves a lot of the issues, but again, I'm not sold on the DNG idea...

Cheers,

Stephen
0 0
robgendreau
How'd you get Spotlight to index XMP? Mine don't show.
0 0
Hybrid
I don't recall doing anything special to make it work - the XMP files are just text files... I have changed computers since the original post and my new system is now running Mavericks 10.9.2 but previously I had OS X 10.8...

I did a quick test now and put a unique keyword in an image and Spotlight picks it up straight away and returns the XMP file.

Which version of OS X do you have? Have you checked that Spotlight isn't being stopped from indexing the location where your files are? Also check that it is returning all document types.

With this setup, I can right click on the search result in Finder and choose "Show Enclosing Folder", which takes me to the XMP file that is normally right below the actual image file. It's ok but still pretty clunky if you have lots of results you want to compare...
0 0
robgendreau
It's irritating. For some reason the UTI of all my sidecar XMPs are org.xmind.openformat.xmp, probably because I had that application, XMind, installed. And it's not a UTI that Spotlight will recognize.

I thought if I changed the "Open with" to a text editor, it would change the UTI. Nope. They show as plain text in kind, however.

So I had to go into the plist of the plugin for rich text for Spotlight and edit it to include the above UTI. Now it works.

Now I need to set something up so it will search the xmp keyword field in HoudahSpot as a template; they seem to be in this format:

   <dc:subject>

    <rdf:Bag>

     <rdf:li>test</rdf:li>

    </rdf:Bag>

   </dc:subject>


Not sure how to do this though, but so far it looks like if you search on ">test<" you'd catch the keyword, or more completely; it looks like normally only numerals would be between ><. And for me, there's also the heirachical keywords in <lr:heirarchicalSubject>, with a format like "<rdf:li>_Locations|UT|Logan Canyon|Guinivah Campground</rdf:li>"

Would there be a way in HoudahSpot to have a template with say <rdf:li></rdf:li>  already entered, or automatically appended, so that you could be sure you're seaching the keyword in xmp?
0 0
houdah
Hi!

When no application claims the .xmp extension, Mac OS X assigns it a dynamic UTI (Universal Type Identifier).
Oddly enough it seems to declare that this UTI conforms to "public.plain-text".
This puts the RichText.mdimporter importer in charge of importing the file into Spotlight.

I am not 100% sure this is the making of Mac OS X. Running mdls on an .xmp file reveals an item kind of "Adobe Photoshop Elements XMP file info file".
It could be that some application – in my case Adobe Photoshop Elements - claimed the .xmp extension but did not provide a UTI.

Searching a .xmp file indexed as text is tricky and may be unreliable.

The real solution would be for an application (or better yet Mac OS X) to claim the xml extension, provide a standardized UTI an supply a Spotlight importer.
This application could also provide a QuickLook importer. Ideally that plug-in would render both XMP content and a thumbnail of the image file.

That's just how I would imagine XMP support in Spotlight and QuickLook to work.
If you have ideas how you would want Mac OS X to handle this, you should voice your opinion at bugreport.apple.com

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
robgendreau
Thanks. It appears to be devilishly difficult to change the UTI; I thought a simple search and replace, but oh no. It's in the resource fork of some files, or was once (not sure if that's still true).

I don't have much hope Apple will do anything; one of the reasons I'm going through all of this is that they don't even support my RAW files (yet...I hope).

At least the text in xmp is structured, so once you're aware of it you can make sense. Photo software does, after all, since it writes it.

As I think I mentioned, though, I think for searching purposes a jpg sidecar of sorts works better. And perhaps things will get better in terms of using DAMs like Aperture or LR to find stuff so it isn't as much of a pain to go back and forth trying to find things.

Rob
0 0
houdah
UTIs are declared by the system and by third party applications.

A third party .app bundle can declare document types in its Info.plist file. For a document type it can declare a UTI and a file extension. It can specify how the UTI fits in the hierarchy of existing UTIs.
I guess XMP should be declared by the system. This can get messed up if two applications try to claim .xmp and map it to a different UTI.

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
robgendreau
I'm still trying to get this sorted out. I can search XMP now, and HoudahSpot will find a RAW file's sidecar with say a keyword called "mom." But in the results you only get the XMP, of course, not the RAW.

I can reveal the results in Path Finder (for some reason a multiple selection will not work to find multiples in the Finder); then I can at least do some stuff in Path Finder with selections to get the RAWs themselves selected. But that work around only works when they are in a single folder.

Is there a way to select just the file names, without the extensions, in results? If I could take the file name minus the suffix and then do another search, adding the criteria "image," I could then find the associated RAW files.
0 0
houdah
Hi!

You may be able to achieve what you want by

Copying file search results as tab delimited text
Building shell commands using a text editor or a spreadsheet

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0
robgendreau
I didn't notice a means to search for a list of file names I could paste into HoudahSpot; is there such a function? I know I can do that in EasyFind but it would be easier to stay with HoudahSpot. I found a shell script but since the results were also in Terminal it didn't quite get me where I wanted. Thanks for the help.
0 0
houdah
Hi!

There is no way to search for a list of file names.

There is "Find by Example". But that accepts only a single file.

Best,
Pierre Bernard
Houdah Software s.à r.l.
Houdah Software s. à r. l.
https://www.houdah.com

HoudahGeo: One-stop photo geocoding
HoudahSpot: Advanced file search utility
Tembo: Easy and effective file search
0 0