Recently Foley and co-workers [Science 1999, 285, 1902-1905] synthesized nanoporous carbon membranes using ultrasonic deposition, which showed an ideal selectivity of O2 over N2 ranging from 3-30 depending on the membrane synthesis conditions. However, as N2 and O2 have very similar molecular sizes and energetics, the reason for this large difference in pure component permeabilities is not well understood. We demonstrate using atomistic simulations of N2 and O2 in single wall carbon nanotubes with constrictions that even the very small difference in the molecular sizes of N2 and O2 is sufficient enough to provide large sieving resistance to nitrogen in nanoporous carbon materials. These findings also suggest the possibility of designing carbon nanotube-based membranes for highly effective gas separation. The effectiveness of a membrane-based separation is bound by the trade-off between selectivity and permeability. Various independent simulations and experimental studies have shown that smooth and slightly corrugated walls of carbon nanotubes result in large diffusion coefficient and hence high permeability of the adsorbates. Our simulations suggest that by the rational design of carbon nanotube membranes that also have the feature of molecular sieving, it may be possible to develop highly selective membranes with large permeabilities.