The roles of various soluble factors in promoting the osteogenic differentiation of adult mesenchymal stem cells (MSCs) have been widely studied, but little is known about how the extracellular matrix (ECM) instructs the phenotypic transition between growth and differentiation. To investigate this question, we cultured MSCs on purified vitronectin or type-I collagen, and used alkaline phosphatase activity and matrix mineralization as indicators of the early and late stages of osteogenesis respectively. Both substrates supported differentiation, but the mechanism was substrate-dependent. Specifically, osteogenesis on vitronectin correlated with enhanced focal adhesion formation, the activation of focal adhesion kinase (FAK) and paxillin, and the diminished activation of extracellular signal-regulated kinase (ERK) and phosphatidylinositol-3 kinase (PI3K) pathways. By contrast, MSCs on type-I collagen exhibited reduced focal adhesion formation, the reduced activation of FAK and paxillin, and increased activation of ERK and PI3K. Inhibition of ERK and FAK blocked mineral deposition on both substrates, suggesting that the observed differences in signaling pathways ultimately converge to the same cell fate. Understanding these mechanistic differences is essential to predictably control the osteogenic differentiation of MSCs and widen their use in regenerative medicine.