How to Avoid Usurious Rates for Online Zotero Storage
April 7, 2010 at 4:26 pm Drew Massey 19 comments
So, my good friend Matthias Roeder was singing the praises of Zotero as citation management software. I’m in the process of finishing up one project and starting some other ones, so it seemed like as good a time as any to contemplate these basic tools of doing research.
I knew that I wanted something slightly more robust than what I had used to organize my dissertation, which was basically lots of word files, and Google Desktop to search them all. While this was more or less effective, and with carefully named digital photos of all of my archival materials things could be easily located, I knew that there was a more robust way to do it. But at the same time, I knew that I had been scarred by Some Bibliographic Management Software [[link to endnote]] which never quite formatted things the way I wanted.
So I’m happy to see that Zotero is easy to use, connects well with Firefox and existing databases like Jstor and Hollis. But the real power with Zotero is the ability to hang files on to bibliographic citations. So you can associate an image or a set of images with a citation for an archival source. Making things even more powerful, you can store it all online, so that your research files are available regardless of where you are working.
All this sounds well and good, right? Well, it is. It’s more than that, it is revolutionary to the way that humanists will be able to work. But there is a catch: consider the following cost for storing information with Zotero:
| Storage Quota | Annual Cost |
| 100 MB | Free |
| 1 GB | $20 |
| 5 GB | $60 |
| 10 GB | $100 |
| 25 GB | $240 |
Consider then the fact that Zotero proudly announces that they use Amazon’s Simple Storage Service. Consider the standard US pricing for that: $0.15/GB/month, with all transfer in and out free until at least 30 June 2010. In other words, it costs $1.80 (not counting bandwidth) to store a gigabyte of your own stuff on S3; Zotero is charging 10 times the price. Hmm.
Luckily, you can use JungleDisk’s legacy support of the WebDAV protocol to skip the Zotero middleman and simply store your files on S3 at cost. Here’s how:
Step 1: Get JungleDisk. If you were really clever and got in on the ground floor, you have a lifetime membership like I do. Even if you didn’t, you can download and install the Desktop edition for $3/month. And the first 5 GB are free.
Step 2: Set up an Amazon S3 account. This is easy to do if you have ever bought anything through Amazon.
Step 3: Create a “zotero” online disk with JungleDisk.
Step 4: Configure Zotero to store your files via jungleDisk. In Zotero, go to Actions > Preferences > Sync. Then click “Sync attachment files in My Library using “WebDAV.” Your url is as follows:
http://localhost:2667/<name-of-your-zotero-online-disk-here>
Do not enter a username or password. You may have to restart firefox once or twice before it recognizes the S3 online disk.
Voila! You are now storing all of your Zotero files in a fault-tolerant distributed system, which is available on any computer where you have jungleDisk and Zotero installed, at a fraction of the price of Zotero’s offering. Granted, you can’t view your files through the Zotero web interface, but maybe some enterprising developer will write an online connector…
Entry filed under: Drew Massey, technology.
Amusicology 
1. mugsy81 | April 7, 2010 at 6:17 pm
I have an even more streamlined solution— start a free Drop Box account and move your Zotero folder to your Drop Box folder. Then, tell your Zotero that your Drop Box folder is the new location of your Zotero folder. Anything you put on Zotero will automatically sync, in real time, to your Drop Box account without your having to deal with the WebDAV crap.
2. mugsy81 | April 7, 2010 at 6:20 pm
I should add that I have included a lot of big files and haven’t paid a cent with that solution. I suppose I’ll have to change when I get above the free initial amount, but you can always increase your storage for free by referring someone to Dropbox.
3. Drew Massey | April 7, 2010 at 9:55 pm
Hmm… sounds nice and simple – on the other hand I have 26.7 GB of digital pictures that I might like to put into Zotero via S3 – that would be a lot of referrals, wouldn’t it?
4. mugsy81 | April 7, 2010 at 11:18 pm
It would — we need to find a sneaky way to get about 400 referrals — but I bet it could be done.
5. Andy, CloudBerry Lab | April 8, 2010 at 3:18 am
There is another option to backup data to cloud storage powered by Amazon S3. Check out CloudBerry Backup http://cloudberrydrive.com/ . It is one time fee and the rest what you pay for Amazon S3. No extra recurring fees!
6. adam.smith | April 8, 2010 at 10:09 am
note that Drop Box does not do the same as Zotero File Syncing (either through Zotero storage or through WebDAV). Among other things it will completely mess up your database if you access it from two places at the same time – generally I think Drop Box is great as a back-up tool, but I would be weary of recommending it in lieu of Zotero syncing.
Neither WebDAV nor Drop Box allow you to use file sharing within a Zotero group, however. Only Zotero storage currently allows that feature.
I have no association with Zotero or the Corporation for Digital Scholarship, but here is what they have to say about pricing:
http://www.zotero.org/support/storage_faq#why_does_zotero_file_storage_cost_more_than_bob_s_discount_cloud_service
7. Drew Massey | April 8, 2010 at 2:20 pm
@adam.smith: Yes, I am aware that there are additional layers of software that sit between Zotero’s web site and S3. What confuses me is that it seems as if they are allowing people to use the actual software for free (up to 100MB), creating the misleading impression that it is the S3 storage that is the cost they are passing along to the user.
One solution that would serve the interests of the Zotero community better would be if they used Amazon DevPay, and billed users as they go for storage. Even if they charged $0.15/GB-Month (i.e., “at cost” for most individual users) the large user base for Zotero would probably bump Zotero as a whole up to a different pricing tier (which tops out at $0.05/GB-Month) That is, after all, the raison d’etre for services like DevPay that expose AWS’s metering infrastructure for cloud developers.
8. adam.smith | April 8, 2010 at 2:36 pm
well, you can suggest it to the folks at Zotero.
My sense is that they have the (imho reasonable) idea to let power users and institutions with large demands for storage bear the costs for the support infrastructure, while keeping things free for very small time users and people who just want to try it out.
Note that the idea of the (non-profit) Corporation for Digital Scholarship seems to be to make this type of support longtime financially sustainable
http://www.zotero.org/blog/building-a-sustainable-zotero-project/
I’m not really taking any position here and certainly not arguing with you, just passing this on – if you disagree with these choices, get in touch with the people at GMU or post on their forum.
My main point, as someone who spends a good amount of time trying to help people on the Zotero forum, was to strongly caution against using drop-box instead of syncing and to point to the limitations of using WebDAV as opposed to Zotero storage.
9. Trevor | April 8, 2010 at 3:59 pm
Glad to see you are so excited about Zotero! I’m Zotero’s Community Lead, and I would like to respond to your concerns regarding Zotero File Storage.
First, you’re not really cutting out “the middleman” by choosing JungleDisk. In fact, you’re just substituting a different, and arguably less valuable, middleman. While JungleDisk is charging you $3 per month to provide a front end to Amazon S3, Zotero File Storage offers a full-service application built around S3. Indeed, beyond providing access to your files through the Zotero website, Zotero File Storage also provides other functionality. For example, Zotero File Storage is essential if you want to share files within a Zotero group, since technical limitations prevent the use of WebDAV. And of course Zotero File Storage is also much easier to set up, and because it does not require the use of third-party software its compatibility is guaranteed.
Second, your pricing analysis is rather skewed, since you omit transfer fees, which are hardly insignificant. Moreover, you don’t include the cost of JungleDisk. For example, to get 5 GB of “free” storage from JungleDisk, one needs to pay $36 per year, plus transfer costs. Zotero in comparison will provide 5 GB of storage for $60 per year. So right away we’re talking about a differential of $24, not the order of magnitude you suggest. What does that $24 buy you? In addition to transfer costs, it provides access to individual and group sharing functionality which can’t be provided via WebDAV. It provides one-click setup. And perhaps most important, it helps to support development of Zotero, software which you already love.
At Zotero we’ve always been committed to providing users with a wide range of options to access and store their data, but we hope you’ll agree Zotero File Storage not only provides a valuable technical service, but that it does so at a fair cost.
10. mugsy81 | April 8, 2010 at 11:26 pm
Adam, can you explain this comment about Dropbox further?
“Among other things it will completely mess up your database if you access it from two places at the same time”?
Thanks!
11. mugsy81 | April 10, 2010 at 10:39 am
I don’t totally follow the pricing issue anymore — maybe a new chart needs to be made. But I find that I can upgrade my Dropbox to 50GB for $100/year, which, combined with the fact that I still see no drawbacks to the Dropbox option, seems like it could be the best move.
12. adam.smith | April 11, 2010 at 9:38 am
““Among other things it will completely mess up your database if you access it from two places at the same time”?”
Imagine I have my laptop and my desktop computer online at the same time. Both have FF open, with the Zotero folder in a dropbox folder.
I do something in Zotero on my laptop, update the dropbox folder,
then move over to my desktop and maybe do something on FF there – already I might have compromised my database integrity.
That’s not a fictional account – things like that happen to users.
The other downside is that it doesn’t allow for sharing in a group.
13. PoorPuer | April 15, 2010 at 10:04 pm
Trevor, thank you for coming to this forum and sharing you take on Zotero’s pricing . I have to say though, as much as I like Zotero, your argument does not convince me. Yes, Zotero’s own backup system offers certain benefits not available elsewhere, but nobody asked me if I have any use for those benefits. On the other hand, my whole database is getting close to 5 GB, and I am still at the beginning of my doctoral dissertation. Very soon it will go over 5 GB, and I don’t see how I can possibly justify paying $120 annually, when, as mugsy 81 pointed out, Dropbox offers ten times more storage for less money ($100 a year). And Dropbox is not unique, other services may have different pricing structures (inlcuding transfer fees that you mentioned), but the outcome is the same — $120 a year buys you several times more storage, and the order of magnitude can be an appropriate expression here.
The strongest argument for using Zotero’s own storage rather than any other option on the market is the sync issue, the database potentially compromised by Firefox being open on several computers at the same time. But is this really an unsurmountable obstacle for Zotero developers? Is there no way to address it, or is not being addressed because it keeps the “captive audience” status?
Similarly, one could use Zotero’s backup system for database backup only, and use another service like Dropbox for backing up the storage part. But even the database only part is limited to 100 Mb only, which most active Zotero users reach rather zoon. I’ve been over that amount for a year now, but still have only 250 Mg in my database file. Why can’t I by that much, but have to get this heavily overpriced 1GB at $20?
All this would be understandable if Zotero were a profit-making entity, like the evil EndNote that tried to sue Zotero a while ago or some other corporate greed machine. But if Zotero is not in business of making money for profit, maybe it is better not to force its users to overpay for Zotero’s services and keep them as supportive fans? I’d rather give Zotero the same $120 as a donation from my first post-PhD paycheck (and feel good about it) than have to pay the same out and feel that it was unfair and forced on me because at this point it would be even more expensive to switch to another product.
14. adam.smith | April 15, 2010 at 10:24 pm
poorPuer – two misunderstandings in your post.
The first is related to something I wrote:
“The strongest argument for using Zotero’s own storage rather than any other option on the market is the sync issue, the database potentially compromised by Firefox being open on several computers at the same time. But is this really an unsurmountable obstacle for Zotero developers? Is there no way to address it, or is not being addressed because it keeps the “captive audience” status?”
The database corruption issue exists only with dropbox (or with hosting Zotero on an internally shared drive). It’s not an issue with WebDAV – as the original blogpost suggests using – there are other WebDAV services besides Jungledisk and universities could offer faculty and students WebDAV sevices. So no captive audience, no. The problem with dropbox is not solvable on the developer side though – that’s just the nature of SQL databases.
Second misunderstanding:
“Similarly, one could use Zotero’s backup system for database backup only, and use another service like Dropbox for backing up the storage part. But even the database only part is limited to 100 Mb only, which most active Zotero users reach rather zoon.”
Syncing your database with Zotero does not count towards your storage quota and is free regardless of the size of your database (Zotero currently doesn’t sync databases above. ca. 5000 entries, but that’s a technical limitation). I have a 68MB sqlite database synced, but only use 0.6MB of storage.
So if you don’t need the two specific advantages of using Zotero storage – sharing in groups and one-click set-up – you’re in no way forced to use it in order to have full functionality.
15. SciPlore | April 21, 2010 at 8:53 pm
hi, maybe you would also like our software “SciPlore MindMapping” http://www.sciplore.org/software/sciplore_mindmapping/ which combines PDF management (including bookmark import) and mind mapping with your existing reference management software (e.g. JabRef/BibTeX, Mendeley and to some extent Zotero). on our website is also a video which gives a short demonstration.
16. Michael A | June 29, 2010 at 12:36 pm
For those of you who do not necessarily need to sync your attached files (.pdf, .jpg, etc.) I would also recommend making copies of your notes/references using the “generate report” button. Personally, I use the report feature to make an .html file that I then immediately sync to evernote using the evernote firefox add-on. I can then use access my notes on any computer. Granted, this is not a “set it and forget it” alternative, but with a couple clicks it makes my ability to access notes immeasurably easier.
17. Stephen McGovern | August 16, 2010 at 9:29 am
Have yuo looked at sugarsync? I think it would solve your problems very neatly. There is a review of it here: http://www.savvyitsolutions.com/os/os-r-sugarsync.html
18. Simon | December 20, 2010 at 4:47 am
Disclaimer: I am a Zotero developer, although I don’t work on sync or ZFS.
As adamsmith notes above, Zotero File Storage is intended to provide a way to sustain Zotero development in the future. That’s why it costs more than plain S3 data storage. As Zotero is a non-profit enterprise, no one is making any money off of it. All revenue goes into Zotero development. Whether this is a compelling reason to pay for it, as opposed to paying less for a third-party commercial file storage service,, is up to you.
Attempting to synchronize Zotero libraries between computers using Dropbox and similar services leads to database corruption because these services aren’t intended to synchronize databases. Even if we were in the business of developing database engines (which we’re not; Zotero uses the database functionality built into Firefox), making this work would be a difficult task.
I think we’ve made a reasonable effort to ensure that users have an alternative to ZFS in the form of WebDAV. The only feature not available to WebDAV users is group file storage, which is again for purely technical reasons. Handling permissions and credentials for WebDAV across groups in a way that is both secure and transparent is not feasible. If you have some idea about how to do this, you are free to play around with the publicly available Zotero source code, but we have thought about how to do this in the past and could not find a good solution.
19. Russ | December 23, 2011 at 8:20 pm
This is old, but *exactly* what I was looking for – Thanks