Recently in General Reference Management Category

Why you should care about APIs

| 2 Comments
Chances are pretty good that you have not really given much thought to Application Programming Interfaces (or APIs) before.  And this is for good reason -- your field of study probably has little to do with computer technology.  But I am hoping you will spend a couple of minutes reading this post to see why APIs actually do matter to you and how they could make your life easier and more productive.

Today, we released Sente 5.7.9.  There were a number of small changes in this release, but the primary reason we had to get this out is because Google changed the structure of the HTML in their Scholar pages and these changes broke our reference detector code (the code that puts the little target icons next to each reference).  So, for the last couple of days Sente users have been unable to get references from Google Scholar pages.

Sente 5.7.9 fixes this problem, but the underlying problem is the lack of a stable API to Google Scholar (and to many other resources).

So, what exactly is an API?  It is an interface for use not by people, but by programs.  A user interface, or UI, is designed to be used by a person. An API is designed to be used by other programs.

An example might help.  If you use Apple's Mail program and you have a MobileMe account, you can access your email either through MobileMe's web-based user interface, or you can read your email in Mail.  The Mail application uses several APIs at MobileMe to get and send mail (e.g., SMTP, IMAP, POP).  If Mail could not use these APIs, it would be forced to try to extract mail messages from the web interface, which would be very difficult, slow and error prone.  And, every time someone at MobileMe tweaked the way the web page looked, the Mail application would have to be modified to handle the changes.

This is where we are with respect to Google Scholar and hundreds of other data sources.  Our targeted browsing code looks at the structure of each web page that it gets and tries to find references embedded within the page. This feature can be very useful, but it is not nearly as useful as it could be if each site supported some standard APIs for searching and retrieving reference data.  And, every time one of the supported sites changes their HTML, we have to modify Sente to handle the change.

Contrast this with PubMed.  PubMed has supported a well-defined, relatively stable API for accessing reference data (and other types of data) for many years now.  This means that Sente can do more things, more quickly and more reliably, with PubMed than we can with most other data sources.  And our users in the bio-medical sciences benefit directly from this.

People in the humanities often accuse developers of academic reference managers like Sente of caring more about biology than about French literature, or medieval art.  This is not the case.  The problem is that the data sources in the humanities (and many other fields) are far, far behind the data sources in biology.  

Instead of treating citation data (titles, author names, abstracts, publication details, etc.) as a little advertisement that will attract readers to their publication, many publishers in the humanities (and other fields) treat citation data as an asset that they can license to services like Thomson and EBSCOhost, who in turn sell access to institutions.  I think it is fair to say that the primary motivation for this way of doing things is not to provide the best support possible for academic research.  (Note that I am not talking about free access to the full text of all articles; just to the basic citation data about each reference.)

Now, you might be thinking that I have confused two issues -- APIs and the licensing of reference data -- and, to some extent I have.  But they are related.  Google provides an API for regular Google searches.  But not for Google Scholar.  Why?  Because the academic publishers would not agree to this because they were afraid of losing their income from licensing their data to resellers.  Thus, we are reduced to trying to keep up with each little change in the HTML that Google makes to their Scholar search results page.

So, back to the original question: why should you care about APIs?  Because the lack of support for open, stable APIs to basic reference data in many fields is holding back everyone  those fields.  We will continue to find ways to make it easier to search the literature in all fields of academic study, but we would be more successful if the publishers actually wanted this to work.

Many of you have, or will have, significant influence on one or more of the academic publishers in your fields.  You may be on the editorial board for a journal.  You decide which journals you submit your manuscripts to.  I think it is time for everyone to use whatever influence they have to push publishers in all academic fields to follow the lead of publishers in the bio-medical sciences.  It is in everyone's best interest (even the publishers) that there be ready access, through both user and programming interfaces, to all of the basic citation data in all academic fields.

Sente 6: Synchronized Libraries, Part Two

| 5 Comments
In my previous post on synchronized libraries I talked about the needs that we were trying to address in our design of this new feature.  In this post, I thought I would let you see synchronized libraries in action.  I will be demonstrating synchronized libraries using a pre-release version of Sente 6.

The following video demonstrates the use of synchronized libraries on one computer.  The mechanism is exactly the same when the copies are on different computers, but this was the best way to show the updates as they happen.


In the real world, performance will be a bit slower than in the video because you will be further from the servers than we are, but updates will still propagate in just a few seconds.  The biggest difference will be seen when first synchronizing a large library and when propagating PDFs -- which are much larger than typical reference updates -- these will take quite a bit longer than in the video.  But other than when you first create a synchronized library, the volume of data going back-and-forth will be relatively modest, so you should still be pleased with the performance.

Synchronized libraries is working in Sente 6, Preview 3, which was recently distributed to our small group of testers.  We will be making one or more public previews available in the coming weeks, so everyone will be able to kick the tires shortly.

Please let us know what you think -- we want to make sure that our design for synchronized libraries works well for as many people as possible.

Michael

Sente and Dropbox

| 8 Comments
Dropbox (www.getdropbox.com) is a new service that makes it easy to synchronize files across multiple computers.  We know that some Sente users have attempted to use Dropbox to synchronize Sente libraries across multiple computers, but we do NOT recommend this.

The problem with Dropbox is that it is primarily designed to synchronize files like word processor documents or spreadsheets, not database files.  Using Dropbox to synchronize database files is likely to result in corrupted files.

The problem is certainly not limited to Dropbox.  Any system that copies Sente libraries from one computer to another must only be used when Sente is not running on either computer.  When Sente is running on the source computer, it is always possible that Sente will modify the file as it is being read by the replication software, resulting in an inconsistent copy.  And if you replace a database file on the destination computer while Sente is running, all of the information that Sente has cached in memory will be out of sync with what is in the newly updated file.  In either case, the likelihood of damaging the database is very high.

So, please use mechanisms like Dropbox with Sente only with great care. 

Obviously, we know that synchronizing your Sente libraries across computers is a very high priority for many users and we are exploring mechanisms to do this.  We expect to be able to say more about this in the relatively near future, but until then, please be careful using tools like Dropbox for this purpose.

Michael

Improved Note Taking

| 5 Comments
I would like to say a few words about our latest release, which includes a major update to Sente's note-taking features.

Since the release of version 5.0 in 2007, Sente has included the ability to attach multiple notes to each reference. (Earlier versions restricted you to a single note per reference.) Each note included a title, page range, quotation and comments.

For many users, this has been a valuable feature, and they mention it regularly in posts about Sente. But we have never been particularly fond of Sente's interface for creating and editing notes. It worked, but it was neither attractive nor terribly functional.

In Sente 5.7, we have completely rewritten the notes interface and we are much happier with the results.  In the new interface, it is easy to see more than one note at a time, and we make better use of limited screen space.  Also, it is now possible to copy one or more notes from within the Notes view, and then to paste them into most other applications.

We have also added a feature that makes it simple to create notes from selections in PDFs or web pages. You can select either text or any arbitrary region in either a PDF or a web page, and then, if you click on the Create Note button, Sente will automatically create a new note for the current reference using the selection as the quotation.  Sente will also fill in the page numbers and add a placeholder title for you.

This ability to create notes from PDFs is not intended as a replacement for tools that let you mark-up a PDF, like Preview and Skim.  If what you want to accomplish is to highlight portions of a PDF so that the next time you read the document you will see these annotations, then we would recommend that you use one of these other tools.  (Annotations created in Preview, unlike those created in Skim, will show up in the Links view within Sente, so you will not need to return to Preview to view them.)

These changes should make Sente's notes feature more useful to many of our users.  I would encourage you to watch the new video on our web site and then to download the latest version and give it a try.

Sente Undergraduate Edition

| No Comments
We have just released the Undergraduate Edition of Sente and I wanted to say a few words about this new version of Sente.

Since we first launched Sente back in 2004, our focus has been on providing the best reference manager for the most demanding users.  This has lead to the development of features like standing searches, hierarchical libraries, targeted browsing and automatic PDF import, to name a few.  During this time, the growth in the use of Sente by graduate students, professors and research professionals has been dramatic.

But we have heard from many undergraduates over the years who really like Sente's features and intuitive user interface, but who cannot justify the cost of the software because they will only use it to gather the references for a handful of papers each year.  These users are not really interested in setting up standing searches or importing large collections of PDF files; they just want an alternative to EndNote.

To make Sente more affordable for this type of user, we are now offering Sente Undergraduate Edition, which has all of the features of Sente, but has a limit of 250 references per library.  This limitation, while too restrictive for most graduate students and researchers, is more than enough for most undergraduate papers.

Sente Undergraduate Edition is not limited in any way other than the size of the library that can be created.

Sente Undergraduate Edition is available now from Kagi for $34.95.  The Undergraduate Edition can be upgraded to the regular academic version for $64.95.

The New Bibliography Fields View

| 3 Comments
I would like to say a few words introducing a new feature in Sente: the Bibliography Fields view in the reference editor.  Here is a screen shot of the new view:

Picture 7b.png

The new view has three major sections.

The top section is where you select one of your favorite bibliography formats.  I would normally recommend that you select the style that you use most often.  Here, the selected style is "APA 5."

The first field in the next section is the reference type.  The value selected in this pop-up determines which fields are displayed in the rest of this section.  In the screen shot above, the selected type is "journal article" so the fields relevant to journal articles are displayed.  There may be data in fields that are not shown here, but fields that are not shown have no effect on the bibliography entry for the selected reference type.

For many reference type / bibliography format combinations, there are fields that are not used in the selected format, but are used in some other common format.  These fields appear below a label that begins "Items below this point..."  While you may want to enter values for one or more of these fields, the data will not appear in the bibliography entry for this reference in the selected format.

The bottom section of the Bibliography Fields view is the preview of the selected reference in the selected format.  This preview is kept up-to-date with changes to the fields above as you leave each field after making a change.  This lets you see just how each reference will appear in the final output in your documents.  (Obviously, the final results may differ slightly if, for example, the selected style calls for repeated authors to be replaced with three dashes.  The preview here is how the reference would appear if it were the only reference in the bibliography.)

The important point to remember about this new view is that the fields that are displayed are based entirely on the reference type.  This list of fields is then prioritized based on the selected bibliography format to place at the bottom any of the fields that are not actually used in the format for the selected reference type.

If you are a Sente user who gets most of your references from PubMed, this new feature will not make a big difference to you.  On the other hand, if you are in the humanities or any field where you regularly deal with many different types of references, and you regularly deal with complex bibliography formats, you should find this new view very helpful as you add references to your library.