The crustal components in a magma plumbing system often contain mechanically heterogeneous layers and structural discontinuities such as pre-existing fracture (PEF) systems. Such layers and discontinuities, depending upon their mechanical properties, can either facilitate upward magma movement or inhibit it at some depths by acting as stress barriers. This study considers a visco-plastic crustal rheology to study the variation of yield strength around the two lateral tips of an elliptical magma chamber (MC), aided by the presence of a PEF, by employing a finite element modelling (FEM) approach. The orientation of the PEF is varied in the FE models and the patterns of localized tensile stress have been examined from these models. The model results find no difference between the yield strength at the two lateral tips of the MC, when there is no PEF in the crustal domain. On the other hand, the presence of a PEF significantly decreases the yield strength around the associated MC tip. We also observed that the yield strength around the MC is also dependent upon the vertical and horizontal separation between the MC and the PEF. When a heterogeneous layer above the MC is taken into consideration, the yield strength depends on whether the PEF terminates above, within, or below the heterogeneous layer. The model results show that the difference in the magnitude of yield strength between the two MC tips is least when the PEF is located above the heterogeneous layer, whereas, it is maximum when the PEF is located below the MC. These results provide finer interpretation in understanding the kinematic evolution of a magma plumbing system, with large structural discontinuities housed within shallow crustal depths.