Phyllite is composed mainly of tiny grains of mica minerals such as muscovite, or sericite. Fine-grained quartz and feldspar are often abundant in phyllite. These mineral grains are usually smaller than what can be easily seen with the unaided eye. They form through the low-grade metamorphic transformation of clay minerals.

Crystals of other metamorphic minerals such as andalusite, biotite, cordierite, garnet, and staurolite might also form within phyllite. Their crystals often grow large enough to be seen and identified with the unaided eye. These larger crystals are known as porphyroblasts. When organic-rich shale is the initial protolith of phyllite, the organic materials are often transformed into graphite. Many phyllites contain enough mica to give them a black color and a submetallic luster.