I’ve changed my Lightroom workflow fairly radically this summer. In this article, I talk about some of the motivations for making this change, how I made the new choices I’ve made, some of the alternatives I considered, and why I rejected them.
In addition to my work as a developer on the Lightroom team, I take the opportunity to teach Lightroom in various workshop contexts whenever I can and I’ve been doing that for close to three years. One of the biggest challenges I see students face is how to establish good file management practices.
In this article, I share my current practices here not because I want to preach that this is the One True Way to Organize Your Photos, but to provide a starting point for you in setting up your own Lightroom workflow. If this fits as a whole, great! If not, pick and choose what works for you and adapt the rest to meet your needs.
I wrote a previous article on this blog titled “How I Organize My Catalog and Why” in October of last year. A lot has changed since then, so I’ll start by talking about what prompted those changes:
So, What Didn’t Work?
The primary complaints that I had with my old system were:
- I wanted to have my best photography with me in a way that I could get to easily. With all of my photography in One Big Catalog, it was far too large to keep on a laptop hard drive. For a while I experimented with having an “archive” catalog where the less-than-best photos went to be forgotten, but selecting the “less-than-best” photos always seemed like a depressing experience and so it felt like a chore, not something I looked forward to doing.
- I wanted to avoid showing my less-than-best photography. Another disadvantage to having everything in One Big Catalog is that it’s hard in Lightroom to browse a catalog and maintain a focus on “best” work. Sure you can create smart collections, etc., but as soon as you click on any other source in the left side of Lightroom’s Library module, that focus goes away.
- I was spending too much time placing photos in folders based on location. My previous system required me to identify the location where a photo was taken before it could take its rightful place in the folders panel. When I travel and shoot in many locations in one day, this becomes a major effort. The folder naming system I used then didn’t respond well when I occasionally “discovered” additional photos taken on an existing day. This often happened when I merged in photos taken by my wife on her iPhone, often days or weeks after the fact.
Overview: What Goes Where and When
You might have guessed from my use of the phrase “One Big Catalog” above that I’ve branched out a bit. In daily practice, I now have four catalogs:
- An incubator catalog where I import new photos, make decisions about which ones I want to show, and add metadata.
- An all raw catalog: OK. I do still have “One Big Catalog” and this is it. All photos that survive the first round of edits eventually land here.
- A selects catalog: If I think a photo is especially good (i.e. something I’d publish on the Internet or sell as a print), I make a copy of it in this catalog as well.
- A family selects catalog: This is basically the same as the “selects” catalog, but for photos of my friends and family.
Why do it this way? I spend 90% of my computer time now on a laptop and laptop hard drives are well… smaller than external or desktop computer hard drives. I rarely need access to everything I’ve shot, so this system allows me to keep the photos that are in active use easily accessible (i.e. on a laptop’s built-in hard drive). The “all raw” catalog is stored only on my desktop computer at home (and my various backup locations), which has much larger storage capacity, and is there just in case I need to dig back and find something. In practice, that’s very rare, so this trade-off serves me well.
The other three catalogs are relatively small. I haven’t finished migrating from my old workflow to the new, but in practice, I’m keeping about 1-2% of what I shoot in the selects catalogs. That is small enough to fit easily on current laptop drives.
Getting Started: Importing Photos into the Incubator Catalog
The “incubator” catalog is where photos first enter my workflow. Here’s how the import dialog looks when I use it:
Side note: I’m not quite Chase Jarvis, posting serious photos with my iPhone, but I do take a fair number of family pics with my iPhone and that’s what happened to be available when I made this screen shot. 
Let’s walk through the dialog and we’ll see why I choose each setting:
- File Handling: I usually, but not always, convert to DNG immediately upon import. I prefer DNGs over RAW files because the metadata cannot be separated from the image data. (In other words, I don’t like the requirement that raw files have a separate XMP “sidecar” file sitting next to them.) If I don’t do this conversion immediately, it’s because I’m in a hurry at that time. I always convert to DNG at some point in the workflow; the only question is when do I take the time hit.
- Copy To: This is the root folder where the Incubator catalog lives, just inside the default Pictures folder on my laptop or main computer.
- Organize: As I mentioned earlier, my previous system required me to spend a lot of time organizing into folders based on both date and location. I no longer do that. The built-in organization by date works very well for me.
- Template: My file naming template is a fairly simple one: my initials, an import sequence number, and an image sequence number. This numbering sequence flows back to my film days (when it was roll# and frame#) and helps me ensure that the number never changes. If I were starting fresh, I might use something date-based, but this works well for me.
- Develop Settings: I like to apply the Auto Tone because it tends to find the sweet spot of exposure in my photos.
- Metadata: I apply my copyright information right off the bat using a template that contains my copyright and contact information.
- Keywords: I never apply keywords in the import dialog because there are almost always multiple subjects in my photos.
- Initial Previews: Depending on how much time I have at the moment, I switch between Minimal and 1:1.
Editing: Round 1 – Basic Workflow
The incubator is where photos live until they are completely processed. What defines “completely processed?” Several things. I don’t always perform these steps in the same order, but I always do each step for every photo:
- Throw out the obvious crap. If a photo is badly technically flawed (seriously out of focus, badly exposed, etc.), I throw it out (i.e. delete it) as soon as possible. If it’s blurry today, it’s going to be blurry tomorrow and will still be blurry ten years from now. Don’t waste your time or storage on it.
- Add location metadata. While I can still remember where it is, I mark every photo for location using the Location tagset in the Metadata panel. I fill in Location (if applicable), City, State, Country, and Country Code.
- Geocode. As much as possible, I add GPS track log data to each photo. (See my article, “Geocoding Your Photos with Lightroom and HoudahGeo,” on the Lightroom Journal for a description of that process. I wrote that back in 2007 when we were all using Lightroom 1.2. I’m still following basically the same process today.)
- Convert to DNG (if not done at import time).
- Make a second copy of the DNG file. If I’m at home, I’ll copy to a second computer; if I’m on the road, I’ll copy to an external hard drive. I keep that hard drive as far away as practical from the laptop, just in case the laptop grows legs. Only when I have a second copy do I clear the CF card. I can’t stress the importance of backups enough: Bad Things happen to computer hard drives (theft, media failure, etc.) and they invariably happen at Very Inconvenient Times. Though painful and maybe costly, you can always replace a lost, damaged, or failed computer or camera. The odds that you can recapture what you’ve photographed – not so much.
- Add keywords for people I know. If there are people I know in the photo, I’ll mark them right away. I’m not actively submitting stock photography at this time, so I no longer spend the time on other keywording; that may change again in the future.
I usually try to do all of these steps fairly quickly after the shoot, even if I’m tired or distracted. They don’t require a lot of deep thought or concentration.
Editing: Round 2 – Selecting Photos for Publication
What does require careful thought is deciding what photos make it out into the public eye. I shoot a lot of duplicates and a lot of variations on a theme until I get to something that makes me happy. Typically I share 1-2% of what I shoot; as you can imagine, it takes some effort to make those cuts.
I use color labels to indicate a final decision about each photo’s fate. Once a photo has a color label, I’m stating that I’m done with the editing process for that photo and it’s ready to leave the incubator. The color labels I use are as follows:
Typically I use star ratings as a tool to help me get to the best of the group: I’ll loupe through each photo in a batch and assign one star to the potentially interesting ones. Then I use the Library Filter to show me only the one-star photos …
… and then assign two stars to the photos that stand out from that list. I keep doing this (increasing the number of stars) until I get down to a small number of photos that I feel especially good about. Usually by the time I’m using two or three stars, I’ve got my final set.
On those photos, then I’ll spend the time in the Develop module to polish them as I see fit, and then I’ll assign the green or blue label (“Final Form …”) to those photos.
After I’ve made those cuts, I’ll mark the remaining, non-selected photos with either yellow or red labels depending on how close they came to being selected.
Clearing Out the Incubator
The goal of the incubator catalog is to be a temporary working space until I’ve done my editing and developing work on my photographs. Once I’ve completed that work for a reasonably large batch of photos, it’s time to kick them out of the nest. Usually I do this for several days’ worth of photos at once, so I don’t have to spend as much time on it. If I’m away from home, I’ll let the photos live in the incubator until I’m back home and I’ve had time to complete the edits.
Exporting Selected Photos
First off, I export a copy of the photos I’ve decided to publish to an inbox folder for the Selects catalog. I use the attribute filter to find all of the photos with green labels and then I export to a designated “inbox” folder for that catalog:
Note: I don’t do a lot of Photoshop retouching, but when I do, I export as PSD instead of DNG. While you can create DNG files from Photoshop files, some applications are unable to read those files (even though they can read DNGs created from directly from raw files) and you don’t really save any disk space compared to a flattened PSD file.
Importing Into the Selects Catalog
Once Lightroom has finished exporting those photo, I relaunch it with the Selects catalog and import these photos. I currently arrange them into photos based on copyright registration (have they been filed or not? if so, which registration batch?), but I’m not sure if that structure will last. I also create subject-based collections so I have ready access to highlights from recent trips and events. These are typically smart collections similar to this one:
Setting Up the Personal Selects Catalog
I repeat this process for the blue-labeled photos (personal/family selects) with one variation. In that case, I export full-sized JPEGs instead of DNGs or PSDs. This is because I’m somewhat more permissive in what goes into the personal catalog and it makes sense to me to try to adjust for that by using less disk space per photo.
Moving Photos to All Raw
I then open up the “All Raw” catalog and import all of the photos directly from the incubator catalog using Lightroom’s Import from Catalog feature.
The import from catalog dialog looks like this:
There’s not a lot to say about this, other than I copy the photos to a new location (near the All Raw catalog file itself) when I do this.
Once this is done – and I’m sure I’ve got the new photos safely copied to my backup drive – then I go back to the Incubator and delete all of the photos that I’ve now copied over to the All Raw catalog. (Remember: one of the design goals was to keep each of the catalogs except for All Raw small enough that they could live easily on a laptop hard drive.)
Credits
This approach was heavily influenced by Thomas Hawk (see his article, “My Photography Workflow 2009“) and by the comment thread on last year’s version of this article. Thanks to all of you for giving me some great ideas to consider!
good ideas, you’re still way more workflow-oriented than me though.
I tend to organize incoming images by date, tag with metadata, then export individual catalogs to show people. I like the idea of a more permanent selects catalog, may have to try that.
[...] This post was mentioned on Twitter by LRQ Victoria Bampton and Eric Scouten. Eric Scouten said: How I Organize My #lightroom Catalog and Why (2009 Edition): http://tr.im/yCRz #photog [...]
[...] an article explaining his workflow methodology with Lightroom. He has just updated this with the new 2009 edition, and his workflow has changed quite [...]
I like how you leverage multiple catalogs in this model.
We have some differing approaches to yours that your readers might enjoy:
http://x-equals.com/blog/?p=823
Thanks, Brandon: I think the workflow you describe is great for studio/assignment work.
And for all photographers/computer users, I strongly agree with your recommendations on having automated offsite backup. I’m a big user of S3 myself (via JungleDisk).
(And, BTW, I love the presets that you guys release!)
Eric – am I to understand that you no longer keyword your images using the Controlled Vocabulary?
@Dan: Yes, that’s correct. I still have the CVKC loaded into my library, but I’m not making a consistent effort to keyword with it. If I were doing stock photography actively, I would continue to use it.
Thanks Eric – have been grinding on what to do regarding key wording and have kept putting off. Maybe I will adopt a limited keyword strategy in that I note specific names for people, wildlife, flowers, etc. – just things that have special names.
In the IPTC job field I use a code that I call events – as an example 2008-11 Africa, 2009-09 Sacramento Air Show. It seems that I have no problem now going back to the event and finding what I want.
Will have to give this a little thought. If you happen to have any ideas I am sure I am not the only one that is grinding on this subject.
Thanks again
Dan
[...] Lightroom ist das Tool, was ich einsetze und das meinen gesamten Workflow bestimmt. Daher ist es für mich um so wichtiger das ich Schritte schnell und effektiv ausführen kann. Ich behaupte nicht, dass mein Weg der einzige Weg ist, sondern das ich damit gut zurecht komme. Dennoch bin ich immer daran interessiert, auf die Einstellungen und die Benutzung anderer zu sehen, um vielleicht das ein oder andere in meinem Workflow mit zu übernehmen. Die folgenden 2 Links zeigen wie man es machen könnte. Ich hoffe ihr könnt Englisch: Photography Tips – Efficient Lightroom Workflow – Step by Step Instruction und Lightroom 2 Technique: How I Organize My Catalog and Why (2009 Edition) — Eric Scouten: Blog. [...]
Hey, this is a pretty well thought out plan, good stuff! I’ve been thinking about a way to organize my stuff into multiple catalogs, if I even need it, if I should try it, etc… This is definitely some good food for thought though. It’s been driving me nuts, not having a good system to keep track of pics I like but won’t really publish, pics that make it to flickr and not a blog/publication, etc, but this may help. Thanks!
I’m not sure why you need the small catalog for your laptop.
I use one big catalog and space is never an issue. I just move (inside Lightroom) photos that I don’t need on my laptop to an external drive. To choose which photos to move you can use any way of sorting photos in LR, by colour, rating, keywords, or the system you’ve described above.
So, when I’m on the road have all of my current photos that I work on on my laptop and all others sitting on external drives (plus backups at home and off site) back home. This way I always have all of my photos with me. The photos on the external drive resides in my LR catalog as a thumnail and all photos on my internal drive are of course the real deal.
Now, maybe this works for me, since I just have 32,000 photos in my catalog, as I am pretty routhless when sorting out the keepers in the frst place. Perhaps when I get to 100,000 or so, things start to get out of hand, but so far it works.
As for “familiy-pics” that is a different matter, but I actually still have them in my big catalog. I have been thinking about moving these to a separate catlog, if it gets too big.
Perhaps I should reply with a quick blog post (dlinderyd.blogspot.com) and describe my workflow in more detail.
@dlinderyd: The space issue is not for the catalog, but for the catalog + photo files. I personally like to have them all on the same volume. It’s easier for me to ensure that the correct things move from one computer to another that way.
I’m new to LR2 and already see the problems you are addressing here. One wuestion I have that I can’t figure out is what happens if you move files that are already in a LR catalog from one folder or hard disk to another. LR says it can’t find them, but is there some way to resynchronize LR to the new disk or folder without having to do it manually? Also, I’d like to hear more about your philosophy of converting everything to DNG. Is this now standard practice? Are there many cases where the XMP sidecar gets lost?
Thanks for the updated workflow information.
When you are adding your keywords for friends and family, is this in the Metadata (and which field?) or as an actual keyword? Is your purpose for exporting to a family selects as jpg so that you can then share them easier? Those raw files still go in your Big Raw catalog, yes?
Speaking of, is there a maximum size for the Big Catalog? I’m about to keyword 8 years of digital images!
@Tom: No, if you’re going to move files outside of Lightroom, you will have to inform LR manually about that change.
However, this is typically not as bad as it might sound. Once you have identified the new location for one file, LR will then look for other missing files in the same relative location (i.e. same name and same folder arrangement relative to the photo you’ve just re-located) and match those as well. In most cases, this means you should be able to get LR straightened out in just a few clicks.
@Leslie: I use keywords for each person and I have some organization to them. (For instance, there is a keyword named “my family” that contains separate keywords for each of the members of my immediate family.) There really isn’t a position in the Metadata field that is suitable for this use. (You might use caption but IMHO that precludes the ability to do any kind of structured searching.)
Re: Maximum size: There isn’t a hard maximum, but you are likely to encounter some speed/performance problems if you go beyond 100,000 photos. (See Maximum size of Lightroom Catalog on Adobe Forums.)
Thanks, Eric
Yes, I asked because I couldn’t see a logical place in the Metadata.
I’m still getting to know Lightroom. It looks like you can export a keyword structure. Perhaps I should break up my database, but I’ll still need the same keywords in each.
I wish Lightroom had the same ability Canto’s Cumulus had (I haven’t used in many years.) You could leave the photo database intact, but have multiple catalogs. You could have several catalogs open at once and if a photo belonged in a different catalog, you could just drag it over to the other catalog.
I don’t like the idea of breaking up my actual files into different places. I guess like you, I could have the big catalog and then copies in different catalogs, but this will also then take more storage.
Gets complicated.
Eric,
For your big catalog, do you keyword it or is it mainly as a chronological database?
So, if I understand you right, a picture can potentially end up in three different locations?
i.e. After moving to “All RAW” and deleting the picture from “Incubator”, you decide that this is really good, and worth showing to both your friends and publish online, and thus ends up in
* “All RAW”
* “Family selects”
* “Selects”
To me, that brings up typical problems from DB-theory like keeping stuff synchronized. Say, six months on you decide to edit a picture in “Family selects”. These changes would then have to be replicated in both “All RAW” and “Selects”, if you like to keep the versions alike.
Have I understood you correctly?
I have a problem with this method, too. I really wish it worked like Cumulus where you could have one image database and just drag the reference to the catalog you want it to be in. I just see too many potential problems long-term breaking the photographs up, not to mention having to export and import and keep all that straight.
Me again.
I’ve been mulling this over and reading more online and I’m starting to come around to your thinking, Eric.
I think I do want one big catalog as a chronological, but why do I need all my garden / floral photos in the main catalog or my sporting dog photos? If one day Lightroom goes up in smoke, it wouldn’t be a bad thing to have a modicum of subject organization in my file structure. If there is a slight cross-over, I probably won’t care if I only make edits on my final file and not the big catalog file. If I need to find it again, it’s named by date, so that’s easy.
Hey Eric,
I have similar workflow I described awhile ago at http://bit.ly/4HA2Z. I also use the concept of an Incubator (I call it Incoming) area for staging but my selecting is a bit more limited, it’s either a Pick or not and I aggressively delete images I don’t need. Like you, only a handful make it out to the public, unless it’s photos of my daughter in which case even a blurry image from The Grand Forest is priceless and the grandparents want to see it.
Interesting you don’t use Keywords much, I’ve been slowing add more and more. I’m also a big fan of GPS data as well the IPTC Location information which I use extensively for finding photos. I might try HoudahGeo again, last time I used it I recall having issues.
Finally, great to see a Lr dev using DNG. I have gone back and forth on this but always end up with DNG as the final resting format. One issue that’s concerned me (but has never been an issue) is with XMP there’s little concern about corrupting the image file, if the sidecar file is trashed so be it — with DNG it’s all or nothing, no?
Great meeting you at Geeks.
@Leslie: Re: Keywords: Have you noticed the Import and Export Keywords commands (in the Metadata menu)? This lets you transfer an entire keyword hierarchy from one catalog to another via a plain-text file.
You asked: “For your big catalog, do you keyword it or is it mainly as a chronological database?” At this point, it’s almost entirely chronological. IMHO the relatively limited time I have for organization is better spent choosing the photos that are worth showing than identifying what’s in every photo I’ve captured. If I had infinite time, I’d keyword more, but since I don’t, it’s become an acceptable sacrifice.
@Carl-Erik: Yes, it’s a risk. In practice, I rarely revisit photos after publishing. If I were to re-edit a photo, I would do it on the big computer with the big hard drive in the All Raw catalog and re-push the changes to the Selects catalog.
@Brian: Was great to meet you as well. I’ll be missing a few weeks here due to some upcoming personal travel, but I look forward to rejoining the group in November. As for the question about DNG, yes, it’s all or nothing. Backups are a good idea, as always.
Hi,
very interesting read, especially when I have started thinking about moving all my photographs to a catalog. I am however facing a problem no application seems to be solving (so far). It might be that Lightroom offers a solution but I have not discovered it yet. I would gladly educate myself if anyone can suggest a solution for following…
I have collected many raw files in last 8 years or so. I usually delete raw files I am not interested in so I do not keep all of them. The ones I keep I process manually using Canon DPP + Photoshop (one by one, very tedious and old fashioned approach) and I save the processed photos as high resolution TIFFs or JPGs. This is because I obviously don’t want to loose all the time I spend on processing the photos and I regularly use these files as input to print.
For each event (holidays, wedding …) I’ve got a directory with a date and short description. Underneath I’ve got two main directories – one where I keep original raws and the other where I keep the hi-res versions. I’ve also got another directory for web-sized versions and another one for slides which are downsized versions of processed hi-res versions – just for running slideshows a bit faster – but these two directories are not important now.
Now, with Lightroom when I import a directory I end-up having two versions (original & hires) in the catalog as individual photos. I do not want either to process originals again or hide originals and work with hi-res only. Some photos I’ve got only original version of.
I would like to be able to associate (I guess manually) originals with their hi-res versions so Lightroom understands them as one object or as one photo having two versions.
When you process a photo in Lightroom you can see the history with all the changes. When association I suggest above is done I guess it would be good to implement it as an history step and Lightroom would stop showing both of the photos individually but would be showing the most recent version (which would be the hi-res version for me). If needed one would be able to delete it and start from scratch with the original. However in library or slideshow it would be shown as one photo.
By doing that I would be able to have both originals and processed versions of my photos and it would behave as if I processed them in Lightroom. I would not need to label originals and hi-res ones and keep filtering out one or the other all the time. Also, for photos I do not have a processed version of I would want to see the original (two separate catalogs with originals and hires photos would not help then).
I am not too sure whether such concept goes against Lightroom philosophy or not however I know quiet a few people around me trying to solve the same problem. We all started ages ago working with directory structure similar to what I described and now is difficult to move away from it.
Does it all make sense?
Thanks for any suggestions!
Regards,
Petr
Hi
Thank you for a very useful article and also to the other contributors that have explained their workflow. I have started using LR2 in the last year having only used iphoto before that. I am also recently new to photography and am slowly getting to grips with raw,tiff dng etc. (20 years computing experience)
Having read all the articles I have gone back to my computer (Macbook and two workstations for home and office) and thought it might be useful to share with you (because you are on the LR team!) my current experience in relation to storage and backup. I am not happy with it and I feel I am having to go through a huge learning curve which is not what LR should be about. I suspect you guys come from a background and experience level that is way beyond mine and many others. You may be making assumptions about knowledge levels that just are not there in many cases.
Here is an outline of my current workflow
My macbook is where I first import all photos perform pick and rejects and then transfer to workstation for developing.
To keep the work done on the workstation in sync with the macbook I export from the macbook as a catalogue and import this catalogue into the workstation (with a usb stick). When I am finished processing I then reverse the procedure by exporting the catalogue from the workstation and importing it into the macbook.
I expect this at the least is giving me a backup copy on both of the computers.
I have also periodically (which means not very often!) performed a time machine backup on my macbook and also copied photos from my workstation to a external storage device.
I currently have 508 photos on my macbook.
However……
1/
On my macbook when I now examine my Pictures file I find 3 copies of every photo under different subfiles
Pictures/Lightroom
Pictures/2009
Pictures/3star sept
No idea why but it must be gobbling up space?
2/On my macbook many of the previews are blank with message “File img xxxx is offline or missing” I have no idea where these are but assume I will see them if I attach the external storage device. However, I would have expected to have been given some clue as to where they are?
In my Folders I have two subsections..macbook and external storage. I can see that 109 of my photos have been moved to external storage. The balance should be on my macbook.
Also on my macbook, some of my previews have question marks (?) to indicate that they have been moved. Here at least i can see a preview of the photo but cannot do any work on it until I go find the photo. When control click to find the folder I am given a box asking me where to look……I have no idea!!!!
This does not appear very clever to me. I usually end up performing a spotlight search for the number associated with the photo and track it down that way. (Very tedious!). Sometimes, I also notice that this will end up at a photo in iphoto that is NOT the photo i want. Then I am completely lost. Where could the photo have gone?? When i do find the photo it is very neat for all the nearby photos to also appear.
3/ To TIFF or Not to Tiff?
Simple question here but it probably sums up my total confusion.
I import raw and can see my photo as CR2. If I export as a tiff to my desktop (or elsewhere) do I still have the CR2 file somewhere?
You guys that have all this figured out can probably point to my infantile errors with ease. However, I would bet that there are hundreds if not thousands of LR2 users like me that are getting totally lost as to what is happening.
You don’t need to respond to this. I just thought it would be useful for you to see how confused some people (me) are getting! Thankfully I am facing up to this issue before I end up with 5,000 photos and a complete mess.
LR is a great product and i will eventually figure it out.
Regards
pierce
I agree with Pierce comments, LR should make our life easier than this. LR is a fine tool for editing, but as a organizational tool, a nightmare for all but the geeks or those who dont have day jobs and can spend the hours figuring all this out. I have got into one helluva mess with catalogues and folders and missing files. I need to start again, so will dump eveything in a safe place, clean up and start again. Ouch.
Maybe when I retire I will have time to figure all this out and eventually bacome more productive.
Hi Eric,
The task I’m doing now is to orgnanize a a large existing collection of images into Lightroom. I thought about it and made a mental plan on how I would like to orginaize it and then searched throughout the web to find out how to do it. You’re the lucky winner of the highly coveted “You explained well almost what I want to do” award. Well done!
Now, I’m trying to figure out how to make the changes to your steps to fit my needs and I have not found that yet…
The idea, like you, is to create catalogs by use (best images, travel, family, etc) and maybe (its potential size makes me reluctant) one for everything. However I am importing from folders that have all that mixed together, so what I would like to to import into an “Everything” or an “Incubator” catalog and mark the images according to use (so far so good).
When I open one of the specific catalogs (say Family) I really want to import the images by attribute (say a specific color) but I only found a way to import a partial catalog by folder. I realize your workflow does not need this because you keep exporting files and reimporting but I would really like to avoid that. I already have my images where I want them (with 3-way replication and geographic distribution and MD5 validation), so I really do not want to create more copies of the images. Even just the time impact of copying to reimport seems highly undesirable, even to just “move” the files in the specific catalogs to the same location as the “Everything” one (the move would be a delete of course).
Do you know of a way to accomplish this or the equivalent? If not in LR2.5, would that be possible in LR3? Maybe a third-party tool or APIs for doing this programatically?
Thanks in advance,
- Itai
neoluminance.com
@Petr: This is not a typical Lightroom workflow. If the original and hi-res versions were in the same folder, you could stack them, which would have the effect (normally) of hiding any photos that were not the stack leader. Given your current layout, you can’t use stacks – all members of a stack must be in the same folder.
I guess what I’d suggest is to have separate catalogs for the originals and the rendered photos and take the time to generate rendered photos for those where you don’t already have them.
@Brian:
1> I’m not sure what’s happening here either, but I’d look closely at the settings you’re using when doing the catalog import/export. It sounds like the photos are not getting matched up properly, or maybe when you’re going back to the laptop, you’re not importing into the same catalog.
2> If you click on the “?” icon next to any of the photos, you’ll see a “Locate Missing Photo” dialog and it should include in the header the location where we last saw the photo.
3> Export is non-destructive. Exporting a TIFF (or whatever) will not affect the source file.
In general, we’re aware of the pain involved in moving photos from one computer to another.
@Itai: Closest thing I can suggest is to import the entire folder into each catalog and then delete the photos that don’t match the criteria for each catalog. You’d probably want to take the time to optimize the catalog (File > Catalog Settings) if you wind up removing large numbers of photos.
@Eric: Thanks. I did think about that (importing everything in multiple catalogs) but what I did not link about that is that these catalogs would not share the same metadata (rating, flag and keywords mostly) because Lightroom is used for creating these and AFAIK those are per catalog.
The ideal workflow for me would be that Import from Catalog would let me choose (smart) collections just like it allows to choose which folders to import.
Maybe something can be done to move the metadata from one catalog to the next using an intermediate representation? The only restriction I have is that most images are on read-only media so that information cannot be in the same folder structure as the images. Is that possible?
Thanks again!
@Itai: There are a couple of ways to transfer metadata from one catalog to another.
1. In Lightroom, choose Metadata > Save Metadata to Files. This will update the XMP in each selected photo to match what’s in the catalog. Most metadata can be stored here in addition to the catalog. Rating and keywords can be transferred this way, but flags can not.
2. You can use Import from Catalog and Export to Catalog, which will transfer all information that Lightroom has about each photo. As you mention, Import from Catalog can not address smart collections. (Good suggestion, and I’ll pass it along to the team.)
There is a workaround, albeit somewhat inconvenient: In the catalog you’re transferring from, select the smart collection you want to transfer. You can export the contents of that smart collection as a temporary catalog. Then open your destination catalog and import the temporary catalog. You can delete the temporary catalog after you’re done transferring.
Eric,
I really appreciate your article and found some very useful ideas. I have loosely been incorporating parts of your workflow already, but as my catalog grows I’m finding a stronger need for an improved organizational workflow. Originally I sought some methods by which pro/semi-pro photographers organize their portfolio work and their family photos. It seems that my work collides here (although I do sometimes get a select photo from those family shots).
While I understand the inherent differences between a LR catalog and it’s folders, smart-folders and other organizational tools within the catalog, I have to agree with some of the other comments in that it seems extremely cumbersome to have export and import just to move files from one catalog to another. I tend to like the idea of a file/catalog browser within LR where your catalogs are all listed in the navigator (perhaps greyed out if they’re currently offline), and photos can easily be dragged and dropped from one catalog to another. Consequently, a catalog can be opened simply by clicking on it. This, perhaps isn’t much different on the surface than a foldering system except for the fact of the performance and storage aspects happening behind the scenes where the images from a particular catalog aren’t loaded until selected.
PS – I initially purchased ($200) Aperture with my iMac a few years ago. I was so disappointed with it that I tried the Lightroom demo as soon as it was available. It was so much better that I purchased it immediately (another $200!) and never looked back.
Thanks again,
Scott
I’m waiting for password-protected photos.
i.e. create a collection and assign a password to it.
Photos included in this collection would appear in LR only if
this collection is “opened”…
I want to strongly support what Scott says about catalogs. I know I’ve mentioned this before, but I really wish Lightroom worked more like Canto’s Cumulus. It was extremely, extremely effective and easy to be able to have several catalogs open at once and be able to drag images to different catalogs. File stayed put in one database, just the reference in the catalog changes. I don’t think I’ll be completely happy with LR until this is resolved.
I don’t remember who mentioned it above, but LR is fantastic for the editing / developing features but the organization of actual files – limit to 1 catalog open at a time is cumbersome and asking for mistakes to happen.
Perhaps I just don’t get the big picture yet. still learning, but that’s my take.
Thanks Eric for a very informative and enlightening post. I think its so highly individual how one build up a workflow, that its really hard to generalize or even give advice on this subject.
A brilliant idea i read in one of the comments is a future wish to be able to deselect folders in a catalog so LR dont cache them by default but rather grey/shadow them out until you instruct LR that you need them. Then you would have them visibly in your folder pane for easy access but would not slow down your performance. Maybe a checkbox next to each catalog could be an idea, where only your checked folders will open and be cached by default. This way you get the best of both worlds, fast access to all your files but only the necessary folders opened/cached.
I surely understand that many users are reluctant to work with multiple catalogs.
I sat down the other night with 4 colleagues and this subject came up. We all shoot more or less the same type of events but still all of us had completely different worklflows.
What we all agreed on was that one big main catalog is madness (long term) and practically impossible if you work on laptops. We also agreed on having separate catalogs for private and commercial images. After this everyone had different opinion and methods which all work fine but it was very interesting and we all picked up ideas from each other. Some sort by year, some by category etc.
I like your idea having an “entry” catalog where all your new shoots goes into before deciding their final destination. I work like this and every single image me and my family members shoot actually pass this catalog for sorting, renaming, tagging and back up.
Scott, Leslie, Michael: While I’m obviously limited in what I can say about future plans for Lightroom, please know that we hear you loud and clear. My own experience as outlined above would obviously benefit from a simpler multi-catalog workflow.
The discussion of catalog organization is very thought provoking. I am using the new LR3 BETA (and I’m a first time user) and I have two different problems: 1) When I imported my photos from my PC to my Mac w/ LR3, they were all JPEGs so I didn’t know that I should probably import them as DNGs. So I imported them as JPEGs and after import of 5300 odd photo files I went to the Mac hard drive and navigated to the Pictures file and found there were 10,300 odd photo files. So, wow, I started looking to see what was going on and found there were 2 JPEGs for each photo; one regular file and one labeled the same but with a -2 after it. So I said, let me start deleting the -2s and save some hard drive space. When I tried to open a file with the -2 deleted, it came up at what looked like 1/2 resolution…very compressed & blocked up–unusable. So I reinstalled the -2s and all was well again. Is this really necessary? Would I be better off space-wise if I imported them all as DNG files? I’m really confused by this one.
2) I did some moving of files around from one folder to another and even though I did it in LR, one batch of files have disappeared from LR. I can find them on the HD and I can open them in PREVIEW, and I’ve gone through the process of pointing LR to the files but it still won’t see them and when I try to open that FOLDER in LR, I get a gray screen with the message “No photos in selected Folder.” What am I missing here? I have a feeling this one is simple.
Chuck
Ok, I got #2 solved. It’s kinda counter-intuitive because you open up the dialog box where you select the files you want to recognize and they are all greyed out and it wants you to select but you can’t highlight them so you just ckick SELECT and it did take them.
As to #1, maybe I could delete all the -2s as when I open the FOLDERS in LR, it shows duplicates of each photo. Is there a way to do a DELETE ALL *.JPEG-2? I don’t want to have to go through and do over 5000 manually. Or should I delete them all and re-import them all as DNGs? That may be the best solution.
Chuck
I wrote back to Chuck privately by e-mail earlier, but thought I would share the answer to his most recent question (how to delete -2.jpg files). When viewing the library grid for All Photographs (use a different source if you want to limit the scope of the search), call up the search tool (Cmd-F). In the filter bar, set the popup menus as follows: select Filename (instead of the usual All Searchable Text), Ends With (instead of the usual Contains All), and then enter the text “-2.jpg”. Now you can Select All and Delete photos.
One thing I am missing in LR2 is a “find missing folders” and/or “find missing files.
Now I have to (slowly, since I need to wait for the screen refresh) scroll through 15k+ images to see whether or not some are missing. Might not need it often, but sometimes I do.
Another thing I’m missing: If I use Expression media to write IPTC into a CR2 file (not xmp), most gets read by LR. Not so if I do the same for a 1D .tif file (Older Raw format).
Last thing: (remotely connected): Would be nice to be able to script IPTC keywording: For instance copy the filename into an IPTC field.
Those three are the main reason I still need Expression Media alongside LR2.
I’m sure you covered this, but I’m either missing it, or just not quite understanding it. I’m looking to do a setup that is nearly identical to yours. Slightly different folder layouts, but that seems to be irrelevant. Where I get lost on your setup, is how you are taking all the photos from your laptop, ending up with all of those on the desktop, and just your selects on the laptop. I’m with you on exporting to the selects, but then I am just not seeing how you end up with all the RAW’s on the desktop. Help please?
Hi Justin, I’ll admit that I’ve glossed over that question a bit. The reason is that I use a source-control system for moving files from one computer to another. (See my article on Perforce as a backup strategy from 2006.)
I don’t say too much about this these days because I think Perforce is too difficult to administer and understand for most users. (If you are a programmer or hard-core computer geek, you would be the potential exception to that rule.)
In place of that, I would look for software that can keep an entire folder tree in sync across two or more computers. I’m sorry to say that I don’t have any specific recommendations here because I’ve been using Perforce in that role for many years.
Eric-Thank you much for your reply.
I think I have finally figured out the answer I was looking for, though. My problem was I tried to read all these tutorials before I started working with Lightroom. As nice as that was with the initial setup ideas-I’ve finally come to realize that I just didn’t understand all of what was being said. There are many parts of this post that I didn’t fully grasp, until I had played with the software a bit more.
Going forward, I will continue to read articles as such, but understand that I might have to toy around before I fully understand what they are getting at.
That being said, I just wanted to let you know that I’m stealing your catalog names, and running with them! RAW, selects, etc-that just perfectly describes the way I wanted to separate my photos. I had a really hard time grasping how the files went from Incubator to RAW, but I finally figured out you import them into RAW, instead of exporting them from Incubator. 3 weeks invested so far, and many more to go before my selects is fully built!
Back again. Eric, I wanted to share a realization based on my own needs to provide a perspective.
After using Lightroom now for around 6 months, I’m looking again at my workflow. Just spent a lot of time looking at multiple sites examining various methods from the All One Catalog zealots to the Catalog for each shoot Camp. I sat down and made a Mind Map of my own needs. I was all ready to start a multiple catalog workflow based fairly closely to yours, when I suddenly realized something.
My initial thinking for multiple catalogs would be: 1. My internally shot “stock parts” textures, objects, etc. I thought, no brainer – that’s one catalog. Then I thought of catalogs for 2. dog photography, 3. family, 4. Fine Art, including, still life, landscape, etc.
I do a lot of Post Production, montages, textures, etc. I also sometimes take an image into Painter to work on. So what happens if I make a photograph of a dog montaged with a Landscape, textures applied and then edited in Painter? (That may sound awful, but turned out quite nice.) That would include 3 different catalogs. Will I be opening and closing catalogs trying to find the images I need for the montage? What then happens to the derivative Montage?
I’ve realized that everyone I’ve been studying does pretty much “straight” photography. I don’t shoot something, process, pick best and then file it. (For the most part.) I want my entire body of work available to pillage and create new visions. I also don’t heavily edit to final choices. I’ve found that often an image that wouldn’t be a final pick on it’s own, works quite nicely for something else. So, If I do one catalog for all of the above, even minus family photos, it would be really large. As much as the “All One Catalog” advocates claim this is fine, it doesn’t make sense. My system is already straining, so what happens years from now?
So, I want to put another plea in, Eric please convey the need for ease of use with multiple catalogs. (Look at Cumulus!)
Second, I need to start visiting the sites of those who do photo manipulation to see how they are using Lightroom.
It’s pretty cool to see how many workflows there are out there. It really is all about what works for you. Your workflow gave me a lot of important elements to consider and I’m sure I’ll be using some of it in my own workflow.
Leslie, thanks again for sharing your experiences. It’s very helpful for all of us to hear the different ways in which people use our products as we think about how we’ll adapt and update in the future.
As noted before, I can’t really say much about our future plans, but know that we hear you loud and clear.
How do you use the red “do not show” label? Is there a way to make LR not show a certain color by default? Or do you simply have some sort of Smart Collection that shows all labels except the red one?
“Do Not Show” does not have any special meaning to Lightroom. I only use it as my own convention to steer me away from photos that don’t represent me well.
I’ve been using Lightroom for about a year. Now that I’ve upgraded to 3.0, I’m assessing again my workflow principles. I’ve designed and built very large data schemas for other domains (healthcare), which has helped me develop a means for designing a schema in Lightroom. Generally speaking, everything has one thing in common and that’s time. In the case of a photo, we can start with creation time. I like having one large catalog, within that there are time periods. Years such as 2010, 2009. Within those there can be subfolders with various titles today’s date, a shooting locale, whatever. The folders really just serve as a browsing tool for me.
The big deal are the keywords. This is where the action is. Every photo gets extensive tagging using a convention. For me, every photo has a shoot date, a shooting venue, name of the photographer, a job number/name, a generic subject (e.g. “race cars”), and information specific to the image (i.e. “Jim Johnson”, “BMW”, “victory lap”).
For sorting, editing, and work flow, I’ll assign individual photos with flags, ratings and colored labels and filter the results into collections.
Let’s say, I want to view all photos of race cars. I’ll do a keyword text search for “race cars”. This will find all photos of race cars, including toy race cars for example. I’ll mark these according to what I’m interested in, and then move them into a collection for editing. I have various collections for things I want ready access to, like “my favorite shots”. Very regularly, I clear out the photos from collections in order to keep things tidy.
Another thing that is very important for me, and the reason for the subfolders based on date in the catalog is that it is the same structure as the folders on my harddrive. This comes in handy when backing up the photos themselves (not the Lightroom backup, the actual harddrive backup). I made this mistake at the beginning, which was to have files from all over my harddrive(s) in no particular structure reliant on being organized by Lightroom and it’s catalog. This made backing up the photos and recovering photos hit-or-miss and cumbersome. So, in short, I’ve found that the most useful thing for me is to start with constructing a very clear structure for the harddrive(s), and stick to it, and build the Lightroom catalog off of that. Using this method, I can easily put my finger on the physical location of a file on the harddrive(s), as well as the logical location and structure it belongs to in the overall scheme of my photos and how to manipulate them.
Eric,
One Lightroom version and nearly a year later, how have you modified your workflow and thoughts on photo organization since this updated (2009) posting?
Jeremy
I haven’t blogged about it yet because it’s still evolving for me a bit. (As you might imagine, my own photography took kind of a back seat to the final preparations to get software ready to go.)
For the most part, my workflow is similar to what I described in this article, but since LR3 added the ability to lock the filter settings across collections (i.e. only browse the green-labeled photos), I’m experimenting with no longer using the separate Selects catalogs.
I would also like to know if your workflow has changed with LR3.
I’m a new user to LR and I’m having problems with not being able to open the catalogs. Finally, I just moved them to a backup drive and have been waiting to find the time to figure out the problem. Thus this post… Now that I’ve moved them to another drive, how do I redirect LR to find them? Then, how do I get LR to OPEN the darn catalogs? I’m about ready to pull my hair out and any help would be greatly appreciated.
If you’re in Lightroom, you can choose Open Catalog from the File menu. Or, when you start Lightroom, you can hold down the Option key (Mac) or Alt key (Win) and it will ask you on startup to find the catalog you want to use.
Eric, thanks a lot for your blog. It helped me a lot with my workflow. So did you change your workflow now a lot or still similiar to this one?
Eric,
Isn’t it about time to start a new Blog for Lightroom 3?
My main reason for not using DNG’s is the online back-ups. I tried DNG’s for a short while, but used jungle Disk as well. Adding one keyword meant having the entire file reupload as the entire file was marked as changed.
Not sure if current online services will do block-level uploading of changed parts, but my internet connection is limited enough to rather re-upload a 10kb xmp file than a 25mb DNG one!
Looking forward to a LR3 update as well, I am still not sure about the most practical way, besides just having one big catalogue, as the filtering and collections make any selection quite easy.
Cheers
[...] are perfectionists who keep only the best. As other strategies have shown, there can be a use for a holding Catalog to hone what you import. This can mitigate against bloat, but it will only hold off for a [...]
[...] [...]
Is this article still valid, being 2011 at the moment? I’m overdue for a complete restructuring of my photo organizing regime, but not quite sure how to go about it.
When I first started, I used Lightroom strictly for my SLR shots, and iPhoto for my iPhone shots. I hated having to use both, so I took everything out of iPhoto and sorted the photos into Processed. And, somehow, I’ve misdeleted a whole wack of photos in the process.
I currently use one catalog at a time, and create a new one when one gets too big.
Raws in the file system are organized automatically through Lightroom by camera then by date.
Processed, ready to go public photos are exported to a “Processed” folder where it’s then divided into a thematic scheme, i.e., People, Places, Flowers, and so on.
iPhone photos first get dumped into a folder, then selected ones go into Lightroom for processing and later exporting to the Processed folder.
This has worked to a point, but has proved to be unscalable. For example, my “iPhone Dump” folder has gotten so big that I never select any to put into Lightroom. Sometimes, I edit photos on the Lightroom, and that photo on it’s own is ready to go public. When I sync with the Mac, that ready photo gets mixed in the iPhone Dump folder, and not Processed. Eventually I would have to sort this out.
Anyways, I’d like to “start from scratch” and adopt the scheme outlined here (as it makes sense) or some likeness to it. How would I go about this? If I reimport everything into a new catalog, would my chronology go haywire?
Eric,.. have you updated/evolved your workflow since this/your post 2 years ago? If so, what have you changed and why?… we, your groupies, are hanging on to your every tweet!
shawn