Frustration and Randomness

Spin systems with strong magnetic interactions might remain disordered avoiding conventional magnetic long-range ordering due to zero-point quantum fluctuations, and supporting a Quantum Spin Liquid (QSL) state. Long-range quantum entanglement in QSLs promotes the emergence of non-local excitations and topological properties, unveiling unexplored quantum phenomena. The immense fundamental interest in this exotic phase of matter evinces the need for new crystalline designs and novel combinations of experimental methods in the study of QSLs. The present PhD thesis focuses on two organic low-dimensional charge-transfer salt families that might host Quantum Spin Liquid states due to geometrical frustration and designedly introduced randomness. Using the combination of multi-frequency high-field Electron Spin Resonance (ESR) and Muon Spin Rotation ($\mu$SR) techniques supplemented by $^1$H Nuclear Magnetic Resonance and transport studies, I propose that organic frustrated magnetic systems are severely affected by the anion layers and by the weak antisymmetric exchange, the so-called Dzyaloshinskii-Moriya (DM) interaction. Amongst crystalline hybrid conductor-rotor structures comprising Brownian rotator anion layers, I survey experimentally four different materials illustrating a variety of magnetic ground states depending on their dimensionalities and interactions with the rotor components. In quasi-one-dimensional weakly coupled spin chain systems, the presence of the controlled disorder originating from the slowing-down of rotators creates a spin-gapped or long-range antiferromagnetically ordered ground state depending on the spatial relation of the anion and the spin-bearing cation layers. Remarkably, inhomogeneous distribution of the disorder of the rotor stopping positions induces a broad distribution of the N'{e}el temperatures at the microscopic length scales. In contrast, two quasi-one-dimensional distorted triangular lattice systems are identified as QSL candidates. In particular, low-temperature zero-field $\mu$SR and high-field ESR measurements in (EDT-TTF-CONH$_2$)$_2^{+}$[BABCO$^{-}$] (EDT-BCO) confirmed the absence of magnetic ordering down to 20 mK despite large, $J/k_{\text{B}}=365$ K nearest-neighbor antiferromagnetic exchange interactions and a moderate anisotropy of $t'/t=1.6$ of the interdimer transfer integrals. The linear field dependence of the ESR linewidth, i.e., the spectral density of the fluctuations, manifests fast spin excitations, reminiscent of motional narrowing. Despite a sizable DM interaction of 0.8 T suggested theoretically, the ESR lineshape remains unchanged down to 1.5 K, indicating the suppression of the effect of DM interaction. Longitudinal-field $\mu$SR measurements reveal one-dimensional diffusive spin transport in EDT-BCO, and predominant fluctuations at the staggered component corresponding to the wave vector of $q=\pi/a$. For comparison, I investigate the antisymmetric exchange in a nearly isotropic triangular lattice organic QSL candidate, $\kappa$-(BEDT-TTF)$_2$Ag$_2$(CN)$_3$ ($\kappa$-Ag, $J/k_{\text{B}}=175$ K, $t'/t=0.97$). The recently discovered sister compound of the first two-dimensional QSL ($\kappa$-(BEDT-TTF)$_2$Cu$_2$(CN)$_3$) displays a spin liquid-superconductor phase transition above a critical pressure of $p_{\text{c}}>0.9$ GPa ($T_{\text{c}}=5.2$ K). In $\kappa$-Ag, multi-frequency and angle-dependent ambient-pressure ESR studies found large antiferromagnetic fluctuations, and a staggered moment of $\mu_{\text{s}}=6\cdot 10^{-3}$ $\mu_{\text{B}}$ at 2 K and at 15 T. Via globally fitting the field- and temperature-dependence of the ESR linewidth using the one-dimensional field theory of Oshikawa and Affleck, I assigned $\mu_{\text{s}}$ to a DM interaction of 1.2 T, weakly suppressed compared to its theoretical value of 5.5 T. Nevertheless, high-pressure multi-frequency ESR measurements reveal that the effect of the DM interaction is completely quenched with a moderate pressure of $p_{\text{DM}}=$0.3 GPa, as a result of moving away from the spiral-ordered phase. Above $p_{\text{DM}}$ Spin systems with strong magnetic interactions might remain disordered avoiding conventional magnetic long-range ordering due to zero-point quantum fluctuations, and supporting a Quantum Spin Liquid (QSL) state. Long-range quantum entanglement in QSLs promotes the emergence of non-local excitations and topological properties, unveiling unexplored quantum phenomena. The immense fundamental interest in this exotic phase of matter evinces the need for new crystalline designs and novel combinations of experimental methods in the study of QSLs. The present PhD thesis focuses on two organic low-dimensional charge-transfer salt families that might host Quantum Spin Liquid states due to geometrical frustration and designedly introduced randomness. Using the combination of multi-frequency high-field Electron Spin Resonance (ESR) and Muon Spin Rotation ($\mu$SR) techniques supplemented by $^1$H Nuclear Magnetic Resonance and transport studies, I propose that organic frustrated magnetic systems are severely affected by the anion layers and by the weak antisymmetric exchange, the so-called Dzyaloshinskii-Moriya (DM) interaction. Amongst crystalline hybrid conductor-rotor structures comprising Brownian rotator anion layers, I survey experimentally four different materials illustrating a variety of magnetic ground states depending on their dimensionalities and interactions with the rotor components. In quasi-one-dimensional weakly coupled spin chain systems, the presence of the controlled disorder originating from the slowing-down of rotators creates a spin-gapped or long-range antiferromagnetically ordered ground state depending on the spatial relation of the anion and the spin-bearing cation layers. Remarkably, inhomogeneous distribution of the disorder of the rotor stopping positions induces a broad distribution of the Néel temperatures at the microscopic length scales. In contrast, two quasi-one-dimensional distorted triangular lattice systems are identified as QSL candidates. In particular, low-temperature zero-field $\mu$SR and high-field ESR measurements in (EDT-TTF-CONH$_2$)$_2^{+}$[BABCO$^{-}$] (EDT-BCO) confirmed the absence of magnetic ordering down to 20 mK despite large, $J/k_{\text{B}}=365$ K nearest-neighbor antiferromagnetic exchange interactions and a moderate anisotropy of $t'/t=1.6$ of the interdimer transfer integrals. The linear field dependence of the ESR linewidth, i.e., the spectral density of the fluctuations, manifests fast spin excitations, reminiscent of motional narrowing. Despite a sizable DM interaction of 0.8 T suggested theoretically, the ESR lineshape remains unchanged down to 1.5 K, indicating the suppression of the effect of DM interaction. Longitudinal-field $\mu$SR measurements reveal one-dimensional diffusive spin transport in EDT-BCO, and predominant fluctuations at the staggered component corresponding to the wave vector of $q=\pi/a$. For comparison, I investigate the antisymmetric exchange in a nearly isotropic triangular lattice organic QSL candidate, $\kappa$-(BEDT-TTF)$_2$Ag$_2$(CN)$_3$ ($\kappa$-Ag, $J/k_{\text{B}}=175$ K, $t'/t=0.97$). The recently discovered sister compound of the first two-dimensional QSL ($\kappa$-(BEDT-TTF)$_2$Cu$_2$(CN)$_3$) displays a spin liquid-superconductor phase transition above a critical pressure of $p_{\text{c}}>0.9$ GPa ($T_{\text{c}}=5.2$ K). In $\kappa$-Ag, multi-frequency and angle-dependent ambient-pressure ESR studies found large antiferromagnetic fluctuations, and a staggered moment of $\mu_{\text{s}}=6\cdot 10^{-3}$ $\mu_{\text{B}}$ at 2 K and at 15 T. Via globally fitting the field- and temperature-dependence of the ESR linewidth using the one-dimensional field theory of Oshikawa and Affleck, I assigned $\mu_{\text{s}}$ to a DM interaction of 1.2 T, weakly suppressed compared to its theoretical value of 5.5 T. Nevertheless, high-pressure multi-frequency ESR measurements reveal that the effect of the DM interaction is completely quenched with a moderate pressure of $p_{\text{DM}}=$0.3 GPa, as a result of moving away from the spiral-ordered phase. Above $p_{\text{DM}}$, a linear field dependence and fast spin fluctuations are found, similarly to the ambient-pressure EDT-BCO. Furthermore, detailed analysis of the high-pressure ESR linewidth gives evidence of the effect of high pressure upon inherent charge fluctuations.