When you delete a file or folder, it by default goes to Recycle bin. From recycle bin you can restore or permanent delete the files / folder. You can also use SHIFT + DEL to permanently delete a file / folder.
But truth is that, its never permanently delete. You can easily recover those deleted files/ folder using any free / paid data recovery software.
So, how to permanently and completely delete a file / folder. You can use a great freebie application Prevent Restore to make deleted files / folder unrecoverable.
How Prevent Restore works?
It just overwrites deleted files with random data, so that deleted files cannot be recovered.