Houdah Software Forum Archive

Houdah Spot Big Sur Apple Mail Search addin isn't loading

Member #1162
Folks, I'm running Big Sur and I can't get the Houdahspot Apple mail to load and run.
Houdah Software
Which version of the HoudahSpot 6 beta are you running?

Where / how does it fail?
E.g. don't you see the plug-in listed in Mail preferences?
Member #1162 original poster
I installed the latest Big Sur update yesterday and the mail app loaded and ran
Member #1162 original poster
I installed the latest Big Sur update and the houdahmail app ran.
Member #1293
Hi, I also use Big sur and apple mail does not appear in the results.

Where can I get the HoudahSpot 6.0b17?

Thanks
Houdah Software
Member #1130
Make sure you can see (and enable) the HoudahSpotMailPlugin in Mail' s Preferences -> General tab and click Manage Plugins... 
It took me a while to get it right.
Houdah Software
@Member #1130 Do you think the instruction in HoudahSpot need improving?

HoudahSpot adds a warning sign next to the "Apple Mail Messages" search location when there is a problem with the Mail plug-in. When you click the warning symbol you will get an explanation of what the problem is: Mail not running, plug-in not installed, plug-in not enabled. 

When the plug-in is not enabled, HoudahSpot lists the steps you should take to enable the plug-in:
"
1. Launch Apple Mail and select Preferences… from the Mail menu
2. Select the General tab
3. Click the Manage Plug-ins… button
4. Make sure that the HoudahSpotMailPlugin checkbox is ticked

If HoudahSpotMailPlugin is not listed or does not stay enabled, try re-installing the plug-in.
"
Member #1130
@Houdah Software
I really like the alert method you are using - the warning symbol with instructions.  Clear and to the point.

It took me a while to get it going because, with the early beta, the Manage Plug-ins button did not appear. So one thing you might want to add is what to do if the button is not there.  But from what you have said to me the more recent betas automatically make sure that it is present and usable - so no need to mention it (I hope).  

@Member #1293 Have you added "Apple Mail messages" as a search location?  Or used the Apple Mail template? The Mail location is not included by default.
Houdah Software
Indeed the "Manage Plug-ins" button does not appear automatically when installing the very first Mail plug-in. Newer betas quit and relaunch the Mail application when no other plug-in is installed. After that the button should show.

In newer betas the "Apple Mail messages" search location is part of the "factory" default search setup. In earlier betas that was not the case. Customized default setups can thus lack the Mail search location.
Member #1294
Hello

I have houdaspot 5.1.6 and mac big sur

houda has full disk access granted
 but houda does not find any email whatsover...no Email within Mail library, but also no EML, EMLX, MBOX that I have stored outside of Mail library. Spoligth has no issue finding all these emails.

I see no pluggin button in Mail preferences.

Does Houda 5 find emails in BIG Sur or only Houda 6?
thank you
Houdah Software
In macOS Catalina, Apple has put Mail searches out of reach of third-party applications. We reported the bug / regression. This was not addressed in macOS Big Sur.

The plug-in is a workaround for that unfortunate situation. The plug-in is a new feature of HoudahSpot 6.
Member #1294
Hi.

Ok that explains how it does not find emails within mail library. But it does not find emails outside of mail library either. I have mbox emails in my documents folders but it does not find those emails.
Funny enough it will find ICS files in those emails attachement but not the emails themselves.
It finds emlx emails but does not find mail authors or recipient .

I just want the same function that I had on catalina.

Am I doing something wrong ?
Houdah Software
I belive Apple has removed the Spotlight importer for .emlx files in macOS Big Sur. The Spotlight will thus no longer get extended metadata (e.fg. authors or recipient) nor text content from .emlx files.

The importer used to be installed at /System/Library/Spotlight/Mail.mdimporter.
You could try to recover it from an older system and install it in /Library/Spotlight or ~/Library/Spotlight
Member #1294
Hello, 

how am I supposed to recover it from an older system... I upgraded to BIG SUR....
- can you share that Mail.mdimporter.  
-Have you tested that adding it back on Big SUr works?

-does this mean that with V5 Houdaspot can no longer find emails with author, the recipient, etc?  
....neither within mail library but also not outside?  It would make V5 on BIGSUR not useful at all for emails.

- Can V6 find emails outside of mail library with author, recipient, date sent etc?

with V5 and Catalina limitation I had to move a lot of emails outside of the mail library, now I need to be able to keep searching within those emails.

thanks
Member #1294
I installed houda 6 beta 35.

It finds emails in mail library (V8) and sees all emlx file with authors, recipient info etc.  but all emlx file outside of mail library are not found.

Is anyone able to search for emlx files outside of mac mail library and do you see recipient, author metadata.

I only use houda of mail search, every day. I did not know that it woudl not work on BIGSUR.

thanks for your help......hopefuly i am just doing something wrong
Houdah Software
Without the Mail.mdimporter Big Sur will not be able to index - and thus make searchable - the metdata you expect from .emlx message files.
Member #1130
I don't think it is that simple.  

I have for some time kept a copy of my mail folders on another volume (daily update with Chronosync).  That has worked well.

What I have done on my just upgraded to BS Mac.

The existing emlx files are still in the Spotlight index with their content indexed.  But new ones are only there with file attributes - the mail content is not indexed.

Check what importers are installed with the Terminal command: mdimport -L
There is no Mail importer (as houdah has said).

I have copied /System/Library/Spotlight/Mail.mdimporter from my Catalina backup to /Library/Spotlight (as houdah has suggested).
After that the Mail importer appears in the list after running mdimport -L

I have then forced reindexing - I did that in Terminal with mdimport -E /Volumes/<name> or you can do it in System Preferences -> Spotlight https://support.apple.com/en-ke/HT201716

Then wait while it reindexes....and finishes.

The emlx files do NOT have their content indexed.

It would seem that the Catalina Mail.mdimporter is not compatible with BS.

Not sure where we go now, except to make sure that all mail messages are stored in Apple Mail so that they can be searched by Mail or by HoudahSpot.  That is fine for me.
Member #1294
@Member #1130, I have done same thing and reached the same conclusions

"Not sure where we go now, except to make sure that all mail messages are stored in Apple Mail so that they can be searched by Mail or by HoudahSpot.  That is fine for me."

last year we were told to move emails out of Library so they could be found by Houda, now we have to put them back into mail library 😂

Houda team, it is strange that this has not been tested..it's kinda of an obvious use-case.....the emails that we were told to move out of mail library last year , are now out of whe the new pluggin can see them.  I know apple is not making it easy and I did add ask for the feature to apple already.
Member #1294
can we copy paste the Mbox back into the V8 mail library or do we need to import through mail appp?
Member #1294
I was able to reimport Mbox back into mail app (they now appear as "on my mac""..I can search with metadata..but the conclusion seems to be that emails stored outside of mail.app cannot be searched.
Member #1294
After some research, it seems that imported mail land to a folder in mail.app called "on my mac".....the problem is that icloud does not backup this folder. If I was to loose my mac, i would be loosing all these old email.... the choice is either have the emails in a folder outside of mac.mail and they will be backuped by icloud but they are not searchable ! or have the emails in mail.app , they are searchable but not backuped by icloud....do I get this rigth?
Member #1130
@Member #1294,  At least we have come to the same conclusion about the mail importer not working.

I am not sure of your backup strategy, but Time Machine backs up your whole mail folder.  Alternatively you can copy/move the emails from "On My Mac" into the folder structure of your email provider and then those will be synchronised (via the IMAP protocol) with the provider's servers.
Member #1130
This was discovered two months ago  https://developer.apple.com/forums/thread/658374.  I don't know if the author of that post submitted feedback to Apple.  I have submitted feedback to Apple (referring only to Spotlight, not HoudahSpot) essentially asking for a working Mail.mdimporter to be reinstated for macOS 11. FB8906806

Just to repeat, this is not directly a HoudahSpot issue because HoudahSpot only ever attempts to use data already within Spotlight indexes. It is Apple's responsibility to get data into Spotlight. Apple really are messing people around with Mail support in Spotlight.
Member #1294
@Member #1130 thank you for the feedback and info.  I have now 95GB of mail archives over the years, I cannot load that into my IMAP provider which asks me to archive after 10GB 🙁

I guess the only solution is to put these emails back into mail.app and rely on timemachine.  thanks for the reminder.
Member #1295
Unfortunately, it looks like the current beta plugin is not working with the current Big Sur beta.  When I upgraded from Catalina to Big Sur, I was still connected to the beta program, so I have the 11.1 Beta (20C5048l).  I downloaded and installed the HoudahSpot beta tonight (Version 6.0b36 (Build 417)), and the mail plugin installs but won't activate. When it restarts mail after activating the plugin, I get the "Incompatible Plug-ins Disabled" message for HoudahSpotMailPlugin.mailbundle - it says to contact the makers for a version compatible with Mail 14.0.
Interestingly, the SpamSieve.mailbundle is working properly (it wasn't at first, but the 2.9.41b1 release fixed the problem).

Not sure if there are any kind of logs that I can provide to help troubleshoot the issue?
Member #1130
All a bit depressing, but I can confirm @Member #1295 issue with the beta of macOS 11.1 

Console has

kernel error:

System Policy: Mail(7653) deny(1) file-write-flags /Users/username/Library/Application Support/Mail/Plug-ins/Bundles/Users/username/Library/Mail/Bundles/HoudahSpotMailPlugin.mailbundle

(and for files within the bundle)

Mail:

Plugin compatibility check failed for HoudahSpotMailPlugin.mailbundle. The plugin's Info.plist must have a Supported11.1PluginCompatibilityUUIDs key whose value is an array of strings that contains the compatibility UUIDs for the supported versions of Mail. The compatibility UUID value is found in the PluginCompatibilityUUID key of Mail's Info.plist

Houdah Software
This was fixes in b38.
That was later pulled because it did not work on Catalina.
A new beta will come today
Member #1295
I just got my new M1 Mac mini, and one of the first things I've installed (after updating to 11.0.1) is the HoudahSpot beta (beta 50).  Unfortunately, the mail integration is still failing even though this is not a beta version of the OS (that's on my MacBook Pro).  HoudahSpot is Version 6.0b50 (Build 431), MacOS is 11.0.1 (20B29).
I'm sorry to have to keep providing bad news, but I'm looking forward to upgrading when this is available (being able to search mail will be great!)
Houdah Software
Compatibility with M1 "Apple Silicon" was just added in HoudahSpot 6.0b53.
Member #1295

Thanks for b53 - it's now loading as expected on my M1 mini.  

My only question is when you search for emails, the "Date Modified" doesn't seem to be able to see the actual date of the email, but rather it's the date when the document was added to disk.  It was confusing because I just added the mail accounts to the machine yesterday when I set it up, so all of the mail documents returned in the search (other than the new mail from today) is showing yesterday's date/time stamp.  I wasn't sure if there's a way for HoudahSpot to read the date of the email and present that in some fashion?

Member #1130
What about "Date Received"?  I have a few like yours.  The "Date Modified" and "Date Created" to refer to the file, but "Date Received", "Author Email Address", "Subject", etc. are what is in the mail metadata.
Member #1294
I have the 6 beta version installed on BIG sur with the plug-in and I can find emails. But I had to import back into mail all the Mbox I had outside mail app. Not the sliding mbox into mail library was not enough, I had to reimport them from mail.

now it works, I can do search with Houda not with email date receive , sent etc. At this stage it less useful that mail search, I will probably not upgrade to v6 when it becomes a paid option.
Houdah Software
You can check View > Details > Info to see what information is available for a given file.
That same information can also be used as search criteria in the Refine pane.

You can actually drag a row (e.g. Date Received) over from "Info" to "Refine" to create a new criterion.