In this paper, a high-order and fast numerical method is investigated for the time-fractional Black-Scholes equation. In order to deal with the typical weak initial singularities of the solution, we construct a finite difference scheme with variable time steps, where the fractional derivative is approximated by the nonuniform Alikhanov formula and the sum-of-exponentials (SOE) technique. In the spatial direction, an average approximation with fourth-order accuracy is employed. The stability and the convergence with second-order in time and fourth-order in space of the proposed scheme are religiously derived by the energy method. Numerical examples are given to demonstrate the theoretical statement.