Category Archives: Mythicsoft

FileLocator Pro version 7.0

I’m really pleased to announce that FileLocator Pro version 7 is finally here!

FileLocator Pro Download Page

It’s hard to believe but it’s ten years since the first version of FileLocator Pro was released back in April 2003. Looking back at the old versions it’s quite amazing to see how much the product has improved over the years and we hope you’ll agree with us that version 7 is another great improvement. But, are the new features compelling enough for you to upgrade again? Well, here are the highlights:

Dedicated Word and Excel Filters

While IFilters work pretty well they don’t always work perfectly. They require a separate install, can often get confused by complicated formatting, and don’t provide text in a format intended for display. Therefore FileLocator Pro now has bespoke filters for Word and Excel files for improved text accuracy and formatting.

As you can see in this simple Excel example it makes quite a difference. However, if you decide you prefer the IFilter format you can revert to it in the Configuration settings.

Document Cache Settings

Document Cache Settings

Document Caching

While FileLocator Pro’s multi-thread searching algorithms are lightning fast they can still be slowed by the need to convert large documents into text every time they’re searched. Version 7 introduces a new Document Caching functionality that allows FileLocator Pro to store converted text in a caching database for use in subsequent searches.

This worked so well for one our beta testers, where search times over 26,000 PDFs went from over 5 minutes down to just 10 seconds, that they were able to use cached searches instead of going with an indexed search solution (avoiding with all the headaches associated with indexing).

Reporting

Reporting has long been a feature we’ve been looking to add to FileLocator Pro. Although it’s always had the ability to print the Hits tab or export to a file any form of customization required quite a strong technical ability. Not anymore, FileLocator Pro now includes a dedicated Reporting tab, with reports for Files, Contents, and Keywords. Each report can be customized and is available in numerous styles and formats, e.g. CSV, HTML, Text etc.

There’s much more we would like to do with Reporting but as always we’ll be guided by your feedback.

Column Filters

If I’m honest this was a self-indulgent feature. Something I really wanted, and actually delayed the release date for. It’s a simple idea, following a search you can filter the files listed by entering filters in the column heading.

Column Filters

Column Filters

Every column can be filtered on the text within the column but date-time and numeric columns can also be filtered on ranges, e.g. > 20KB < 100KB. Try it out, I think you’ll like it.

Saving Workspace

Saving Workspace

Sessions and Workspaces

Working with large result sets can take time, sometimes days. Version 7 adds a new Session/Workspace feature so you can save your search results and then re-load them back into FileLocator Pro at a future time.

It allows you to save just a single search tab (a Session) or all the open search tabs (a Workspace).

Other features

It’s not always the big features that make the biggest difference, sometimes the ‘small’ features can punch well above their weight. Here are some of my favourite new small features:

Search Again Ever found yourself copying text from the Text tab and pasting back into the Containing Text field to do another search? Well now you can simply right-click on the word and click the new Search Again menu option. The search will be re-run with the new word replacing what was in the Containing text field.
LINES Operator If you find yourself only wanting to search the first X lines of each file you’ll love the LINES operator. It can be used to limit the lines a following expression can appear in, e.g.
LINES:3-10 "ERROR"
Include filename in search Prior to version 7.0 FileLocator Pro only searched the contents of a file for the term in the Containing Text field. This new option adds the file name to the contents when evaluating the search.
File name presets
Filename presets

Filename presets

Rather than having to type in *.png;*.bmp;*.jpg;*.tif every time you want to search pictures FileLocator Pro now includes presets for:

  • Folders Only
  • Files Only
  • Music
  • Pictures
  • Videos

Note: With a preset selected the file name field is treated as a Boolean expression.

Thank you

Over the last ten years FileLocator Pro has morphed and grown through great continuous feedback from you, our customers, to the fantastic product it is today. But in some ways we’ve only just started. We’ve literally hundreds of features we’re going to be adding over the NEXT ten years. All of them focused on one thing, helping you work better by finding your data faster and easier!


Update: After this was sent out we had a great question:

“Just please tell me you didn’t do anything to slow it down. FL is bar none the fastest grep utility for Win. Please don’t do anything to bloat it up or slow it down.”.

It’s a good point and one we take very seriously, we understand it’s one of our key selling points. In all aspects of performance from application start-up and core search times to memory foot print we regularly compare the latest version to old versions just to make sure we haven’t inadvertently introduced any issues.

The improved accuracy and layout with the dedicated Word and Excel processing can occasionally result in slightly slower times than with Office IFilters but in our tests it wasn’t significant and was worth the small performance penalty. However, if you want the same Excel/Word processing as v6 just set the IFilter Search Priority to Primary in the IFilter Settings.

While we’re talking about performance, during this v7 release we’re going to be doing some profile guided optimization work to see if we can squeeze out another 5-10% speed improvement. I’ll keep you posted on that work.


Avast Win32:Evo-gen [Susp]: Some customers have been reporting issues with Avast anti-virus flagging the FileLocator Pro download EXE as Suspicious and deleting it, and then blocking any subsequent links to the download. Apart from how this makes us look we can only imagine how nervous it makes our customers feel, not the sort of experience you want for your sparkling new product update.

Unlike Norton, Kaspersky, and AVG Avast don’t have an ISV white-listing program for ISVs to clear their software before publishing and Avast’s support has been painfully slow in responding to our (numerous) requests. For now we can only apologize and thank you for your patience.

30 Sep 2013: We received an email from Avast this morning “False positive detection. We are sorry. Our virus definitions has been updated.”. So, if you’re still having problems please update your virus definitions and try again. If that doesn’t work please contact Tech Support and we’ll look into it.

Agent Ransack 2012 (revisited)

Okay, you may have noticed a distinct lack of Agent Ranasck 2012 in, well, 2012. Unfortunately there was so much work going on with our other products that Agent Ransack work was pushed back.

However, work has started (I’ve even seen a demo) and I promise that it will be out in 2013. Honest. No excuses this time.

P.S. There’s also some really cool stuff in the pipeline for FileLocator Pro and FileLocator Network in 2013. Watch this space!

FileLocator Pro and Large Searches

FileLocator Pro Crash ReportDuring August 2012 we quietly added a new crash reporting module to FileLocator Pro. Based on CrashRpt (an open source product hosted on Google Code) it’s one of the most useful quality control features we’ve ever added, although we hope it’s a ‘feature’ most of our users will never have cause to see.

Since then you may have noticed an increase in memory management related upgrades to FileLocator Pro. It’s not a co-incidence.

We’ve had a slow trickle of crash reports over the last few months and while most were odd, quick to fix, edge-case samples the majority have been related to memory management issues. It didn’t take long to see that FileLocator Pro had a problem on low spec’d machines performing searches where the data was in the gigabyte range and involved millions of files. We found a few problems that were simply bugs in the code, e.g. algorithms that reserved more memory than was necessary, but some of the problems were more subtle function related issues.

By default FileLocator Pro will record up to 10,000 lines of text per file and each line can be up to around 20,000 characters. That’s not usually a problem when searching in a limited set of files. Rarely will a file have 10,000 hits or a line have 20,000 characters. However, when searching over a very large data set with criteria that might not be very selective (e.g. searching for the letter ‘a’ – which was the actual search phrase in one of the crash reports we received) it can be a problem. It can be compounded by searching through file types that may not have EOL (End Of Line) markers, such as EXE or DLLs. Finally to make the whole thing just a little bit trickier, what might be a problem on a scrawny 512MB laptop is not necessarily a problem on sturdy 16GB PC.

The trouble is that FileLocator Pro doesn’t know at the beginning of the search if it’ll find a few hundred files with hits on a few lines (easy), a couple of files with hits on 10,000 lines (not a problem) or a million files with each one reporting hits on 10,000 lines (problem… probably).

FileLocator Pro 6.5 introduces a pre-emptive based solution. Based on the amount of memory installed on the machine FileLocator Pro sets an upper limit for un-restricted results per search (from 20MB up to around 200MB). If during a search that limit is reached FileLocator Pro starts restricting the search. Results for each file are reduced to around 20 lines, with a maximum of 256 characters per line, and the restriction is retained until the search finishes. If the search still runs out of memory then rather than crashing, as it did previously, it terminates the search.

Our tests on very low powered machines with just 512MB have shown a huge improvement in stability for very large searches and so far we haven’t received any memory related crash reports. Job done? Not quite but it’s one more step in cementing FileLocator Pro’s place as the ultimate super fast, rock solid, search and data analysis tool.

In a previous post I talked about ‘pushing a button that I think does nothing’. I hope you can see from our response to these bug reports that when you ‘Push the Button’ and send us a crash report it most certainly does something!

PST and MSG attachment searching

In 2007 Joel Spolsky wrote a blog post about gnarly problems, called Where there’s muck, there’s brass. It basically argued that real benefit to consumers comes in solving gnarly problems not nice simple fun ones.

We’ve just had our own ‘mucky’ experience dealing with attachment searching in PST and MSG files. While the MSG format is nowhere near as complicated as the PST format both have nasty surprises when accessing the attachments.

However, once it was all up and running it was impossible not to have a silly grin watching a demo of FileLocator Pro finding some ‘secret’ text inside a PDF, attached to a MSG file, attached to an email in a PST file, that itself was zipped up and attached to an email in another PST file. How cool is that!

In ‘Other News’ we also have a new Q&A site. It’s the same sort of thing as StackOverflow but just for Mythicsoft products. Check it out: http://qa.mythicsoft.com

Joining the social media party – fashionably late?

Over the next few months we’re going to be adding social-media functionality to the web site. Today’s first small step was to add twitter links but more is planned.

Now I realize that we’re hardly blazing a social-media-engagement trail here but better late than never. Please let us know how you think we’re doing and what we’re missing.

FileLocator Pro version 6 released

We did it! FileLocator Pro 6 is out and it’s looking great, for more information check out the 6.0 release newsletter here:
http://www.mythicsoft.com/Page.aspx?type=flpro&page=rel60

Making major changes to a product is not something to be taken lightly. Most of us have experienced the disappointment of installing a new update to a much-loved product only to find that rather than improving it the result was something bordering on useless.

I remember my personal frustration when upgrading PaintShop Pro from version 7 to 8. What had been a fast loading, easy to use, rock solid application suddenly morphed into a slumbering buggy monster. It was unceremoniously removed from my PC and thrown in the bin. It was only recently, 8 years later, that I felt brave enough to give it another go, now version X3. Sure enough many of the issues have now been resolved and it’s returned as my favourite image editing program but I still remember that sinking sense of disappointment.

Continue reading

New Product – FileLocator Remote

After six months of careful crafting we’ve just announced a really cool new product: FileLocator Remote. I won’t go into too much detail here about what it does, since that information is available here and here, suffice it to say that it is a distributed file searching application.

Looking on the web I can’t see anything that does the same job (please correct me if I’m wrong) and usually that would send alarm bells ringing about a potential problem with the concept. However, this time we didn’t actually come up with the idea, our customers did! After receiving numerous emails asking if FileLocator Pro could do distributed file searching we decided to build something that could.

A beta version of FileLocator Remote is available for download:

Download FileLocator Remote (beta)

You’ll need to be running Windows XP/2003/Vista with .NET 2.0 to run the main Search Manager application but the Search Agent can run on anything from Windows 98 or higher. I’d love to hear your feedback so please drop me a line: dave -at- mythicsoft -dot- com.

FileLocator Pro 64-bit version

We’ve been receiving more and more questions regarding FileLocator Pro on 64-bit platforms (mainly centered around non-working shell extensions) so I finally decided to try a 64-bit compilation of the source. I was pleasantly surprised to find that after just a couple of hours fixing some Visual Studio highlighted problems we had a working 64-bit version of FileLocator Pro.

The only ‘hard’ problem was some cryptographic code that contained hand-written assembly routines. Instead of re-writting this I linked with the excellent CryptoPP library instead.

Since the 64-version was compiled from the current trunk source, rather than the 4.0 787 build source, it’s been labelled as ‘beta’ for now. It’s available here:

Download Page

ps I should probably say that all testing has actually been done on a VMWare virtual machine running 64-bit Windows XP.

What’s so hard about writing software?

I recently received an email which read:

“Hi, I was wondering, if I give you a good idea for your program, and I mean a really good idea for an improvement (and my idea is simple to implement too!) could I get a little royalties from your sales”

Where to start… I think the emailer had failed to understand where the difficulty in writing and deploying software lies. Having ideas is easy, everyday millions of people have ideas, some of them brilliant. Now this may sound a little controversial but unfortunately those ideas in and of themselves are valueless, it is only in the embodiment of the ideas that value starts to be created. Eric Sink talks more about this in his excellent article on Starting Your Own Company:

“Although you may not believe it right now, ideas are essentially worthless. You are emotionally invested in your idea. You’ve spent lots of time convincing yourself and others that the business will work. You are devoted to your idea and you do not want to give it up.

But like it or not, your idea alone is not valuable. In the business world, ideas are worthless. Real value comes from good execution.”

So is the difficult part in writing the actual software? Well, not really. Writing software is, for the most part, fairly easy. Not as easy as having ideas but it’s not that hard to write a small utility, especially if it only needs to run on your machine. Sure, it’s a little harder writing something that will install and run reliably on the Wild West like eco-system we call Windows. But an automated build and test system along with VMWare Workstation goes an awful long way to making this simpler.

Where things get a little tricky is the modification and enhancement of an existing product. The list of enhancement requests for FileLocator Pro is large and there are some real gems of ideas in there. The trouble is that many of them just don’t fit into the basic way FileLocator Pro works. So we often ask the question “How many people will we really annoy by changing the way the product works versus how many people will absolutely love the new features”.

But all those difficulties are nothing compared to the biggest issue. Building something that people will pay for. In the real world most people would not think twice about spending $40 on a night out, many would think that was positively cheap. Yet in the intangible world of consumer software many people will balk at the idea of spending $30 on a piece of software that then can own and use for life.

Now I’m not saying that I’m immune to the cheap-skate mentality when it comes to paying for software. It’s only fairly recently that I’ve realised that paying for the best software rather than going for free alternatives more than pays for itself in the the productivity increases. However, most people will settle for a free mediocre program rather than pay for a great one especially if it’s something they’ll only use occasionally, a sentiment which is not hard to understand.

Anyway, back to my emailer. I tried to explain that I would be pleased to hear about his idea but I wouldn’t be able to offer any sales royalty. He (I assume it was a he) was quick to write back and let me know the foolishness of my ways:

“Anyways I realized how to do it myself, to [sic] bad you don’t want to know what it is.”

I wrote back wishing him all the best, and I truly meant it. Writing software that people will pay for is hard, really hard.