The present invention provides a composition comprising a sustained release layer and a fast release layer. The sustained release layer comprises a water-soluble polymer and a first pharmaceutically active agent. The fast release layer comprises a matrix forming agent and a second pharmaceutically active agent. Generally, the composition provides fast and sustained (or controlled) release of a pharmaceutically active agent for at least 6 hours and preferably for at least 1 to 3 days. The composition may be incorporated into a dosage unit form, such as a vaginal insert. Methods of preparing the composition are also provided.