Aoife has it. Every DevKit has two NPN transistors (or one UMH3N dual) cross-coupled like this:
DTR ──[10k]── base Q1 Q1: collector → EN, emitter → RTS
RTS ──[10k]── base Q2 Q2: collector → IO0, emitter → DTREN or IO0 only gets pulled low when DTR and RTS are in opposite states. When both are asserted together (which is what many serial terminals do when they open the port), nothing happens. esptool toggles them in a sequence that holds IO0 low while EN goes high, so the chip starts in download mode, then releases both to run.
With your direct wires, any change of either line resets or straps the chip, hence the resets when the monitor opens and the stuck download mode.
Also add 10k pull-up on EN plus 1 uF to ground. Without the cap, EN can come up before the 3.3 V rail is stable and you’ll get random boot failures on power-up, especially with the AMS1117 which is slow.