Unmanned Aerial Vehicles (UAVs) are expected to fully exploit their maneuvering capabilities to conduct aggressive maneuvers to complete tasks such as racing and aerobatics even in harsh environments. However, most of related works often focus on slow movements only suitable for regular tasks in well-structured environments. This motivates us to propose a highly robust control scheme designed specifically for UAV aggressive maneuvers, encoded by rapid and large changes in either position or attitude. The core of our approach is the data informed incremental dynamics and the quaternion facilitated control scheme. The former facilitates the model-free control that eliminates the requirement to model the complex dynamics at high velocities and accelerations accurately; While the latter avoids potential singularities during large angular changes. In particular, we first utilize one-step-backward data to construct incremental dynamics, a model-free representation of the UAV dynamics. Then, the constructed incremental dynamics serves as the basis for the development of both position-priority and attitude-priority control schemes, wherein the novel quaternion based aggressive maneuver tracking controllers are designed with complete theoretical analysis. The aggressive maneuvers such as roulette, barrel roll, multiple-flip and cobra maneuvers are chosen for tri/quad/hexa/octocopter platforms to evaluate the performance of our proposed position-priority and attitude-priority control schemes, during which the linear velocity up to 20 m/s, the angular changes up to 360 ^◦.