There are a number of factors that affect lamp life. First off, you are right about “average rated lamp life.” This means that half the lamps burn out faster than expected and half the lamps last longer than expected.
Some bulbs burn out faster when they are installed base up (as in recessed fixtures) instead of base down (as in a table lamp) because heat rises, damaging the components. Bulbs normally burn out faster when they are in an enclosed fixture because the heat builds up inside. This is true for CFLs as well. MaxLite  makes a CFL that is made specifically for enclosed fixtures. LEDs seem to do really well in enclosed fixtures.
Another factor is fluctuations in the electrical current. Sometimes the electricity spikes and causes the bulbs to burn out faster. Many LED lamps can handle these fluctuations very well. You can consider using the 130V versions of most incandescent lamps, which mean they are operating at about 10 percent less than their capacity, allowing them to last longer.
There are also rough-service lamps on the market that hold up to bigger-than-normal vibrations. These are a good option if you have recessed fixtures or ceiling fixtures on the first floor and rambunctious kids (or athletic first dates) on the second floor.