Powerful search tools in Windows & Mac

If you are an information worker (academic), having great support from powerful search tool is  crucial. Unless you have that sharp searching too, you will have trouble to pick that grain of information from the gigantic jungle of information coded in the form of data, sentences, or books.

There are great tools everywhere; but, some stand out in their capabilities than others.

The three giants in the Windows environment you might need to check are:

  1. Dtsearch (Windows)
  2. X1 search (Windows)
  3. Copernic desktop (Windows)
  4. FoxTrot Professional search (Mac)
  5. ? Devonthink (Mac)

 

Personally, I am not that much fond of Copernic mainly because it has no internal previewing tools; and, it seems to consume too much resource of my machine.

My number 1 pick is DtSearch. It is the best in its class in digging the tiniest of information. The proximity search is an invaluable tool to find associated ideas.

X1 comes closer. It is more of a document manager just like Devonthink in the mac than a specific searching tool. X1 also a wonderful application. It is cheaper than DtSearch.

 

As to FoxTrot, it is quite comparable to the DtSearch. But I like the preview system in FoxTrot even more.

The proximity search in DtSearch requires you to write the distances between the words(phrases) explicitly like Mary w/5 John (‘search Mary and John within the distance of 5 words’); while Foxtrot has a little scrolling window to search within a paragraph, within a sentence or less closer phrases.

One might put DT as a competitor to Foxtrot in the mac. But, I think FT is much superior on the search side while DT rocks for its AI and other organizational tools.

(Note, I don’t like giving links to the products because I don’t want to sound that I want to get a penny by associating them to my small, free, blog….I am dropping these notes because I believe these notes might help somebody out there; not because I have some other agenda. I used to keep these notes in my internal system; i put them out now in case somebody get sth useful out of these notes).

Mendeley vs Citavi vs Qiqq (also Sente and Bookends)

I haven’t used my Windows machine for a while now. I was then curious how the reference managers progressed in these periods. I was specially curious about Mendeley because I struggled with that application for some time then.

So, here is my observation: Mendeley stayed the same for the last couple of years. There is no real development; nor any change of any relevant sort since I knew the application. All the icons, the settings, the menus; the features: I see no changes. It is as clumsy as used to be; in many areas. One of the properties that Mendeley sucks at is how reference is downloaded from the internet. It attempts to use Google scholar; combining with its metadata extraction too. What it does is: it attempts to detect some DOI or other identifier to the PDF in the first few pages; and then, use that information to download reference information from Google scholars. For me, the result is a total debacle. It has always been a debacle. Mendeley can detect the papers only less than 5% of the times; as my PDFs don’t typically have metadata information; nor are they always published articles. Many of them are books; or drafts of books, and earlier versions of published articles I received from friends.

My favorite feature of Mendeley, which had been, still is: the BibTex sync feature. I have to admit, I have been tempted to live with Mendeley because of that feature. But, heck, if you have wicked reference data, what is the point of syncing it to Bib file. You will have incomplete citations ultimately. You will be embarrassed in front of colleagues when you realize that your references are incomplete after you sent out the paper. Because of the importance of the feature, I will focus on this feature in comparing the reference managers.

Citavis is not very far better than Mendeley when it comes to reference extraction from the internet. It can even be worse. I was able to download references from the internet only if the book has ISBN numbers or the article has DOI number. Otherwise, manual insertion is the only way I am left with. Look at this tutorial to learn how the process is clumsy in this application: https://youtu.be/MyaW9q_464w?list=PLkLfx87WKrTZTfifTvttqgKwzyQSTDYY5

In Citativ, when you read a PDF file, you can highlight or quote a certain text: comment on it; or give a short title to the comment and the quote. I totally love the idea of giving a short title to the quotation I make from a PDF reading. This feature is also available in Sente. The idea is: you quote a certain sentence or paragraph from the PDF; then, give a title which summarizes the core point of the quote and tag it if you want to. These quotes serve as a short summary of the article. The titles are your reminds of the core points of the quote. It is like summarizing the summary. Very neat approach to reading articles. The neat part in Sente is each of these short quotes could be exported as a separate note file. That means, if you have 20 quotations from the article, you will have 20 short notes: titled appropriately in a folder in finder. The problem with Citavi is each of the quotations are not exportable to separate notes. They can be exported as single file only. That means, it is not any better than reading and annotating a PDF in Acrobat Reader or other PDF readers (PDF exchange; or Foxit in windows: PDF expert in Mac)  and exporting a summary.

Qiqq is very different. Its way of extracting references from Google scholars is comparable to Sente. You click the PDF; click BibTex sniffer: you will be given Google scholars to pick the references. If Qiqq failed to detect the title of the PDF correctly, you can manually select the title. Qiqq immediately populates the scholar search with the selected text.

I have given the following book for all the three reference mangers: It was only Qiqq which correctly imported the full reference information.

gq4omd

I think the reference downloaded in Qiqq is much better than Sente. Sente has an advantage of downloading from multiple sources like WorldCat; Stanford, British libraries…very good results in some sources, weaker results in others. I used to get the most complete reference data from Stanford library website. But Sente sometimes fails to download the Publisher Field from many sources. Bookends can pick from Google Scholar, Justor and two other sources. But the process  of downloading a reference data (called Autocomplete in Bookends; targeted browsing in Sente; BibTex sniffing in Qiqq) is most elegant in Qiqq and Sente.

 

The other interesting feature of Qiqq is the brainstorming feature: absolutely brilliant tools to play with your references. It can also be used to track the positions one author took over time, how his/her ideas change in the long run. it can also be used to study the history of ideas: where a certain phrase appeared first; then, how other authors reflected that phrase in their publications. Look at these tutorials to see how the Brainstorming works in Qiqq:

 

My ratting of these reference manager’s capability of downloading references from the internet:

  1. Mendeley = 4/10
  2. Zotero= 3/10
  3. Citavi = 2/10
  4. Bookends = 7/10
  5. Sente = 9/10
  6. Qiqq = 8/10

Why is Sente higher in this ranking?

Because it offers much better choice than Qiqq on the sources. Qiqq does it elegantly on Google scholar; but, it cannot download from other sources which potentially offer more complete reference data.

Conclusion: if I ever have to move to Windows, I will definitely use Qiqq (in combination with OneNote or ConnectedText).

Curiota: a replacement for nvALT

nvALT has been one of the most useful productivity apps around. I have been using it for the last couple of years. I totally love how it works; and started to use Markdown because of it. It is the best pieces of software to capture flying ideas fast. As time goes by, Brett, the developer who imported it from Notational Velocity, said that nvALT will be not updated, and promised to develop a replacement application. He promised a year earlier; but never released the application yet. In the mean time, as the number of notes I have grew exponentially, I have trouble to manage them in a single folder (which is how nvALT stores the files). Since nvALT has no other  means of organising notes except the tags, organising the notes has been very cumbersome.

I then started to look around a number of other tools. I ultimately get hooked into Tinderbox. But, TB is not the easiest, best application to catch small ideas as they pop-up in my mind. I usually get confused where to put the notes in TB; since my notes are usually organised into Containers (folders). I also tried three other cool applications which are great for the task:

  1. Unclutter
  2. MiniNote Pro
  3. and, finally, Curiota

Curiota, I slowly learned, is the true replacement for nvALT; if you can live with RTF, rather than Markdown. It is a little sister of the well known Curio, mind-mapping and note taking application. I am not that much a fan of Curio. But, I find its little sister very useful. Curiota has all the things nvALT has and more. It also has a means of grouping files; unlike nvALT’s flat filing system. Like nvALT, it transparently puts the files in Dropbox folder, something I highly appreciate. It also has a global shortcut which is used for a snappy note taking. The “stay on top” feature is another little touch, still extremely useful for somebody like who takes notes all day.

Where Bookends sucks

I am now checking out other alternatives as the fate of Sente is looking dismal. The best alternative for the users of Sente looks like Bookends. I think there is some kind of communication between the developers of the two applications. One of the reasons that we suspected the abandonment of Sente came from the developments on the Bookends side. we have seen, the developers of Bookends have been preparing to grab the former users of Sente.In their latest updates, since June 2015, they have been modifying their application to import Sente references. But, still, there are a lot of glitches to move references from Sente.

But, personally, I am more worried about the capabilities of Bookends as a reference manager than the migration. The migration is a work of few days. But, if the application has some fundamental weakness, that will be a pain for a long time; that I am afraid the pain of migrating my references might not worth the effort.

First the strengths:

  1. Bookends seems faster than Sente; at least at the startup.
  2. It works well with a number of other applications such as Devonthink, Tinderbox and Scrivener.
  3. And, most importantly, it has some cool tools called **Global Change**  which seem very useful. These tools helps to make a change to a number of references in one sweep. Sente also has this system; implemented differently.I think the Bookends has an upper hand here. One of the worst footprints of Mendeley from the Windows that remained in my reference for ages was: the Titles of the references were placed in the place of the Journal. I don’t why Mendeley does that. But, the Titles were exported as Journal. I was not able to change that for a number of years within  Sente. Bookends does it in a sweep, just a couple of seconds to fix about 1500 references. Yes, that is great programming.

 

and the issues:

I think bookend is quite good reference manger. But, it has some really deep issues:

  1. The PDF reader is ugly; and not even comparable to the reader in SEnte. Sente gives the best PDF reading experience ever; not even the dedicated PDF readers like PDF Expert, iAnnotate, Acrobat Reader can reach it. Bookends has a mediocre PDF reader.
  2. Not well organized: the tools and features are jumbled here and there. It is really not clear which of the menus do what. At the first look, the app generally looks unattractive. But, honestly, I am less worried about that. I just think some people might not appreciate it. Personally, I just want my job done. I am not going to wear this app for  my birthday party.
  3. But, the real issue, and the true depth of shit of Bookends is on the reference detection and downloading part. The whole focus of the Bookends seems on the MedPub. Generally, most reference managers can detect references from the major databases (search engines) like PubMed and Google Scholar. But, Sente has been efficient in doing it from a broad array of sources: that I cannot list all here. The most important of them, for me have been: Worldcat and  Stanford University Library. Sente made the process perfect by its feature called Targeted Browsing. The two sources offer the cleanest references while Google Scholar gives out the most incomplete metadata. So, when I was trying Bookends, I was hoping that Bookends would do the same.

Assume that I received a PDF book from a friend. I want to download the metadata. In Sente, I would drop the PDF to the library, Sente displays its Citation Lookup  dialogue box in which I will select the Title of the book and choose WorldCat.

joudw

The top 5 sources are the most important. The title of the book, “Italian Syntax…” is going to be automatically pasted in the WordCat. Now, look at the WordCat website.  gku8e

That Red Circle makes the insertion of reference so elegant. Clicking the red button populates the reference information. Worldcat gives a complete reference data; i rarely find a mistake. Note that these 5 sources can be expanded, if required. I used to have a large number of other sources including Stanford University.

Bookends has a similar feature. But, the implementation is inefficient because it is restricted to a few If I want to do the same in Bookends, the process is clunky and inefficient. The sources are not expandable that, if they don’t work for you, you will be stuck. The fact that I cannot grab references from the WorldCat database is really disconcerting to me because that is the number one source for me. All the references for books come from it. It gives the most complete metadata.

 

First, Bookends asks you to attach the PDF. You will get the citation window after  the PDF is attached. That is way step away. After you attach the PDF you will have the following window:

txpz41

This is where I am frustrated. The part I marked with the big red rectangle is supposed to display the Title of  the book. But, it doesn’t. Therefore, if you have to put the title, you are supposed to do it manually. You have remember the title, or copy it before hand. That is strangely sluggish. Second, the sources at the right lower corner  are really useless to me.

Except Google Scholar, the rest are useless, really. I cannot modify or add a new engine either. So, I am stuck. The offer is Take it or leave it. You will be happy if you are a medicine student; fucked otherwise. Google Scholar is quite ok for articles. But, it is one of the most incomplete sources.

Bookends has another method of downloading references using an internal browser. But, I think that one is even worse. I put the title of an article; out of the total of 20 articles in google scholar, which Sente detected all of them, Bookends was able to detect just two.

All in all, I think Bookends is not really polished at downloading metadata.

The difference between Sente and Bookends might seem minimal here, from outside. But, for some one who will use the process thousands of times, even the tinies further step is one more pain. While I like many of the features of the application, I find it hard to adopt the app as my main reference manager  because of  this problem. So, I am contemplating either to stay with Sente to the last breath of the app, or check out other alternatives, Papers 3 probably.

is Sente abandoned?

Sente has been my favorite reference manger for the last couple of years. It has the most elegant reading interface; the annotation and quotation features are incomparable to any other PDF reader, let alone reference manger. I enjoyed every bit of the time I spend with Sente. Importing reference data, and downloading PDF files alongside, has never been as great. Unlike any other reference manager both in windows and mac environment (I have tried many of them), Sente allows downloading references from a very wide variety of sources. Its targeted browsing has been of utmost service for me. I really love how the application is designed; how it all is implemented. Sente is extremely well-crafted application; much better than Papers and Bookends in many aspects.

But, unfortunately, there is no update of any kind from Thirdstreetsoftware for the last few days. They shut down the blog, and stop replying emails. There are also some internal rummers that Sente might not bee see developments. I am truly worried if Sente is vanishing into nonexistence; all the time I spend on organizing my library; all the annotations and notes I made….I don’t know how to live without it. Very sad part of proprietary software;  the end is always ugly.

It also makes me wonder what kind of person would develop such a polished application for years; and ultimately abandon it. They have been developing it for IOS quite recently. There should be something seriously wrong!

 

 

Replace Tinderbox with Scapple

Tinderbox has been very proud of its mapping features. That feature is indeed the main selling point of the application. But, I just realize, one  doesn’t  need to go through the pain of learning an application as complex as Tinderbox to gain these benefits. A little sister of Scrivener, Scapple can do it the mapping of ideas with the smallest fraction of cost; with the easiest pace of learning. It is also better than mind mapping applications mainly because it is portable (that is, you can just drag your notes to Scrivener), and the rigid outlining forced in the mind mapping applications doesn’t exist in Scapple. It is just free, a clean piece of paper. Write, connect in a way you want your ideas to flow.

 

Go and try it. You don’t have to worry about learning a complex, arcane piece of software to graph your ideas.

 

Abandoning Tinderbox and wikis; sticking with plain text

On Note-taking, wikis and Tinderbox

General lessons:

  • Wikis are bad: they will be broken if u move something. So, never use them: history is the best teacher; hyperlinks are stupid for keeping information for a long time
  • Future proof plain text (markdown) files are the best.
  • Keep all the text notes in the same folder, inside Dropbox
  • plain files can be access them by different apps (as the need arises).
  • Use services such as Simplenote and Dropbox to access them via the cloud (syncing)

    these were the steps i used to sync my notes to various apps

  • Create the notes in nvALT
  • Store them in a folder in Dropbox
  • Index the folder in Devonthink
  • Sync nvALT to Simplenote
  • Sync Tinderbox to Simplenote
  • Sync Scrivener to Simplenote

    Work on any of the app, as the need arises. The changes will be reflected everywhere (windows pcs, ipad, iphone, all fetch from the same folder).

  • nvALT is the best for simple notes (drop what is in your head)
  • Tinderbox is best for creating connections of ideas
  • Scrivener is the best for drafting

After some more experiences and lessons, I realize that the syncing between Simplenote and Tinderbox is broken. Tinderbox adds some weird symbols to my text when I sync them to it via Simplenote. I have been waiting for Tinderbox six if it can solve the problem. It turns out that Tinderbox 6 is by no means better than the previous version in this regard. Its arcane nature, its little communication with other applications in Mac still remains a major issue.

Therefore, I am forced to abandon it for now< at least until the formatting problem is solved. (I requested Mark Bernstein, the developer of Tinderbox a number of times, to fix the issue. Even if he promised so many times, I haven’t seen him doing it for over a year now. so, I am losing hope with him too)

Abandoning Tinderbox also invalidates Simplenote’s fuction. so, I am also not using simplenote anymore.

The whole writing system is build around dropbox now (the nvALT) folder. For creating connections between ideas, now, I have to rely on Scrivener and Devonthink (and Scapple for visualization).

now, more simplified steps:

  • Create the notes in nvALT
  • Store them in a folder in Dropbox
  • Index the folder in Devonthink
  • Sync nvALT folder to Scrivener via Folder sync

Simpler and more efficient, at least until i need visualization of my notes (which happens rarely)..