Each year over 6.5 million individuals are estimated to acquire an invasive fungal infection, and approximately half will die as a result – surpassing the combined mortality of malaria, HIV and tuberculosis. Additionally, the number of antifungal compounds is extremely limited, with only a handful of drug classes commonly used. In contrast, there is a much wider range of fungicides available, often with the same mechanism of action as antifungal drugs. As a result, soil dwelling pathogenic fungi, such as Aspergillus fumigatus, have developed antifungal resistance through exposure to fungicides. Large-scale application of fungicides has led to an increasing prevalence of antifungal resistance. The mechanisms of resistance vary amongst populations with different genetic factors contributing towards the phenotype. To assess genetic variation at the population level we constructed the largest filamentous fungal pangenome, to-date, incorporating over 1,000 isolates over a 100-year period, spanning 34 countries. We then investigated contributions from the accessory genome towards drug resistance and defining population structure. Furthermore, using the temporal breadth of sampling, we have been able to calculate a time-corrected phylogeny that identifies the rate of gene gain and loss and how this varies across the population. Additionally, we reveal that specific cohorts of genes undergo differential rates of gain and loss, over evolutionary time periods, revealing the dynamics of the pangenome in A. fumigatus. These results highlight the ability of the species to acquire or lose genetic material, which may enable survival under new selection pressures.