Similar to probe-rs, OpenOCD does not have support for the Xtensa architecture. However, Espressif does maintain a fork of OpenOCD under espressif/openocd-esp32 which has support for Espressif's chips.
Instructions on how to install
openocd-esp32 for your platform can be found in the Espressif documentation.
Once installed, it's as simple as running
openocd with the correct scripts. For chips with the builtin USB JTAG, there is normally a config that will work out of the box, for example on the ESP32-C3:
openocd -f board/esp32c3-builtin.cfg
For other configurations it may require specifying the chip and the interface separately, for example ESP32 with a J-Link:
openocd -f interface/jlink.cfg -f target/esp32.cfg