Random .asp file generated within the web server?

I have a static html website and few pages that use php like forms. On the front end, the website looks fine, however when I try to login through ftp and noticed there are around 189600 files that end with .asp extension being generated within the server.

The file names are all like these:

   aada-oakley-047495.asp
   Aadcaj-puma-22256.asp
   AAeEc-furniture-s1-94186
   .......

Is this a sign of the website being hacked? Should I deleted all these files or leave it as it is?