How to generate an authentication token for limiting access to my server by external queries

I intend to realize a search engine in a niche area where the results must be searchable only from the front page of my website not through API or web scraping by third parties. It is not therefore any kind of token for users authentications, as the access to the website will be public at least in the beginning (no paywall or user accounts involved.)

My question is which method would be less computationally expensive to generate for each user/visit when someone initiates a search. I thought to use local storage of a random token generated at page loading (such as when some bot is scanning the page to not be able to create the token and therefore to not access the API for receiving search results) however in order to check that a token was issued legit (by my server) this means to grow a continuous database storage with all tokens issued earlier and consumed by users.

This not being a practical solutions for a huge number of users when the traffic will increase I want to know if someone used with success something similar or some better approach.

I don't want to use reCaptcha as a validation method [for human users] as this would offer a very poor user experience on the platform, degrading also the speed of using the system to run the searching queries.

The frontend will be made on React or Vue and backend on Python.