INFO How To Protect From Getting Indexed

[abelcustoms]

You all know what an unprotected index is
Well if you don't it looks similar to this

​

What's the problem, you say it's not safe? In short, we can use Google; they will provide you with simple search commands and tools that you can use to target these exposed websites.
You might have, for instance, a very important zip file that you do not want to share. This zip file just happens to be inside your apps folder as you see in the picture. In other words, all you need to do is have a user find your site by using Google to type in that specific command, and boom. The file is now compromised and accessible on the internet. A lot of us use this method to easily access our content and provide it to others. But, in true words, this is not the correct way. To fully protect this site from being found by others you must include an index.html or PHP that will show something instead of an open directory.

I have a solution for you today that I've been using for a while and want you guys to be protected as well.
It not only is an index that closes your directory but still allows your users to search and find the applications you need.
It's not pretty but it works and it's simple. I have made a few edits and custom b64 images inside, to support more than the average files.

Simply place the index.php wherever you want your directory to be protected this means images, files, documents, files inside files you name it.
It's not full protection as it still allows users to find the files (manually) but, hehe it should keep those pesky index searchers out of reach.

To view the content, you need to Sign In or Register.

 

[bc12]

Nice concept.

Along similar lines, I keep a default 'Under Construction' page, but I also incorporate a .htaccess file w/ username & password pre-configured. Anything visiting the domain/folder is prompted for the id & pw.

It's not so much for 'security', but like you said, to deter the bots & admin/admin scripts...

If their invited/expected, they could easily be given the id & pw. On some sites, I've even seen an image posted of the 'password' with a msg stating to type in what you see... ie: banana....

Other simple security things i tried were:
- Changing default installation directory name to something random (ie: wordpress, etc)
- Changing the name of admin folder if possible
- Directory/File permissions should be only what is necessary.
- Disabling remote root SSH access if possible.

But adding the id/pw seemed to work best and dropped my 'visitor' counts to what I expected.

 

[AfterLife]

Nice concept.

Along similar lines, I keep a default 'Under Construction' page, but I also incorporate a .htaccess file w/ username & password pre-configured. Anything visiting the domain/folder is prompted for the id & pw.

It's not so much for 'security', but like you said, to deter the bots & admin/admin scripts...

If their invited/expected, they could easily be given the id & pw. On some sites, I've even seen an image posted of the 'password' with a msg stating to type in what you see... ie: banana....

Other simple security things i tried were:
- Changing default installation directory name to something random (ie: wordpress, etc)
- Changing the name of admin folder if possible
- Directory/File permissions should be only what is necessary.
- Disabling remote root SSH access if possible.

But adding the id/pw seemed to work best and dropped my 'visitor' counts to what I expected.

Care to share the .htaccess file with a default username and password so we can change it please pal

 

[abelcustoms]

Care to share the .htaccess file with a default username and password so we can change it please pal

It's a nice option but not a solution, the issue with modified htaccesses if not setup right can cause more harm than good. In fact, many tools like AnonymousFox inject an attack abusing this method. You are safest to create files and folders that have no generic name and use header to rewrite paths when navigating. My file manager organizes my files however it changes the name of that file in the directory. While still keeping a list of what that file is and location.

 

[bc12]

Care to share the .htaccess file with a default username and password so we can change it please pal

Sure...
Just fyi, I have my own custom server setup and access to the command line.

This is how I do it w/ Ubuntu & Apache... Mileage may vary.

To create website user & password

sudo mkdir /etc/secure
sudo htpasswd -c /etc/secure/.htpasswd username

Configuring Access Control with .htaccess Files
Begin by editing the main Apache configuration file to allow .htaccess files:

sudo nano /etc/apache2/apache2.conf

Find the <Directory> block for the /var/www directory that holds the document root. Turn on .htaccess processing by changing the AllowOverride directive within that block from “None” to “All”:

For example:

<Directory /var/www/>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

Save and close the file when you are finished editing.

Restart apache:

sudo systemctl restart apache2

To protect a domain/folder, put this .htaccess file into the root of the folder.

AuthType Basic
AuthName "restricted area"
AuthUserFile /etc/secure/.htpasswd
require valid-user

To give access to a folder, put this .htaccess file in the folder.

Require all granted

 

[abelcustoms]

Sure...
Just fyi, I have my own custom server setup and access to the command line.

This is how I do it w/ Ubuntu & Apache... Mileage may vary.

To create website user & password

sudo mkdir /etc/secure
sudo htpasswd -c /etc/secure/.htpasswd username

Configuring Access Control with .htaccess Files
Begin by editing the main Apache configuration file to allow .htaccess files:

sudo nano /etc/apache2/apache2.conf

Find the <Directory> block for the /var/www directory that holds the document root. Turn on .htaccess processing by changing the AllowOverride directive within that block from “None” to “All”:

For example:

<Directory /var/www/>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

Save and close the file when you are finished editing.

Restart apache:

sudo systemctl restart apache2

To protect a domain/folder, put this .htaccess file into the root of the folder.

AuthType Basic
AuthName "restricted area"
AuthUserFile /etc/secure/.htpasswd
require valid-user

To give access to a folder, put this .htaccess file in the folder.

Require all granted

I'm cringing seen this be attempted with guys new to terminal.

 

[bc12]

It's a nice option but not a solution, the issue with modified htaccesses if not setup right can cause more harm than good. In fact, many tools like AnonymousFox inject an attack abusing this method. You are safest to create files and folders that have no generic name and use header to rewrite paths when navigating. My file manager organizes my files however it changes the name of that file in the directory. While still keeping a list of what that file is and location.

Crap. Is nothing sacred anymore!? hehe

Thanks for pointing this out to me. I'll have to research it a bit further dammit...

Maybe I'll put a disclaimer on my post... lol

 

[abelcustoms]

Crap. Is nothing sacred anymore!? hehe

Thanks for pointing this out to me. I'll have to research it a bit further dammit...

Maybe I'll put a disclaimer on my post... lol

It's all about security, automation tools that read, find, and inject are ones you want to avoid. Sophisticated tools learn from your server, and find backdoors. Then inject a admin account to both cpanel, WordPress systems. What's worse they keep a ssh access to your server add it to the botnets list. Trust me when I tell you, no public panel in this forum is safe. If the creator knows of a backdoor, so does this script. This is why I made my own panel, i have trust issues. Always will...

 

[bc12]

I'm cringing seen this be attempted with guys new to terminal.

Okay, remember, I did specified this wasn't for 'security' purposes!

I use it solely to "prevent stupid search engines from indexing my site". (and stupid bots).

It wasn't meant as a guide on how to prevent intrusions from hackers using specialized tools like AnonymousFox.

 

[abelcustoms]

Okay, remember, I did specified this wasn't for 'security' purposes!

I use it solely to "prevent stupid search engines from indexing my site". (and stupid bots).

It wasn't meant as a guide on how to prevent intrusions from hackers using specialized tools like AnonymousFox.

I'm just pointing it out, changes to your files in an attempt to close one door can open new doors for others.

 

[bc12]

View attachment 3448
It's all about security, automation tools that read, find, and inject are ones you want to avoid. Sophisticated tools learn from your server, and find backdoors. Then inject a admin account to both cpanel, WordPress systems. What's worse they keep a ssh access to your server add it to the botnets list. Trust me when I tell you, no public panel in this forum is safe. If the creator knows of a backdoor, so does this script. This is why I made my own panel, i have trust issues. Always will...

I get ya. I've seen a few tools posted here that make me "cringe".

But, like I said, the post wasn't trying to secure a server. Just to stop google/bing/etc from indexing a site or it's data. Maybe I misunderstood something as I was just trying to contribute to the subject. Apologies if I misunderstood.

Sounds weird to be proud of, but my domain is nowhere to be found in google.

 

[abelcustoms]

I get ya. I've seen a few tools posted here that make me "cringe".

But, like I said, the post wasn't trying to secure a server. Just to stop google/bing/etc from indexing a site or it's data. Maybe I misunderstood something as I was just trying to contribute to the subject. Apologies if I misunderstood.

Sounds weird to be proud of, but my domain is nowhere to be found in google.

Ha! All my domains can be found. But not my IP.

 

[abelcustoms]

I get ya. I've seen a few tools posted here that make me "cringe".

But, like I said, the post wasn't trying to secure a server. Just to stop google/bing/etc from indexing a site or it's data. Maybe I misunderstood something as I was just trying to contribute to the subject. Apologies if I misunderstood.

Sounds weird to be proud of, but my domain is nowhere to be found in google.

you have the right idea bud, but it's best to educate you on the possible outcomes to approach it in that demeanor. There are ifs and that's, and I mean in no way to belittle your way. I just think a bit of education is necessary when you fiddle with the .htaccess files in an unsafe way. Not saying your way is unsafe, but just take precautions, and try not to attract the wrong crowd. There is a way inside everything you just have to be devoted to it. With the right dedication and research, you have no idea what one person can cause damage to your server. Even Xenforo is not safe. . . look at baibato for example, with 100k+ users and a wiped server, you'd think they have their things together considering they are huge.

 

[bc12]

you have the right idea bud, but it's best to educate you on the possible outcomes to approach it in that demeanor. There are ifs and that's, and I mean in no way to belittle your way. I just think a bit of education is necessary when you fiddle with the .htaccess files in an unsafe way. Not saying your way is unsafe, but just take precautions, and try not to attract the wrong crowd. There is a way inside everything you just have to be devoted to it. With the right dedication and research, you have no idea what one person can cause damage to your server. Even Xenforo is not safe. . . look at baibato for example, with 100k+ users and a wiped server, you'd think they have their things together considering they are huge.

Yeah, I have to admit, my level of experience is minimal. So, I do appreciate the 'heads up' about .htaccess. Nothing worse than having your server 'Owned' and putting everything/everyone at risk...

Regarding precautions...

I've been trying to wrap me head around your statement... "My domains can be found by not my IP"...

Do you mean your hosting your domains offsite on hostinger or something similar? But your home IP is/was always behind a vpn?
or more interestingly, do you host your domains at home and route their traffic to your home IP anonymously?

If #2, how do you do it? generally speaking of course... If you don't mind me asking...

So that's what happened to baibato... I was wondering what happened. I think they got a new owner too if I'm not mistaken... Man, that would really suck!!

Anyway, thanks again, appreciate the tips!

 

[abelcustoms]

Yeah, I have to admit, my level of experience is minimal. So, I do appreciate the 'heads up' about .htaccess. Nothing worse than having your server 'Owned' and putting everything/everyone at risk...

Regarding precautions...

I've been trying to wrap me head around your statement... "My domains can be found by not my IP"...

Do you mean your hosting your domains offsite on hostinger or something similar? But your home IP is/was always behind a vpn?
or more interestingly, do you host your domains at home and route their traffic to your home IP anonymously?

If #2, how do you do it? generally speaking of course... If you don't mind me asking...

So that's what happened to baibato... I was wondering what happened. I think they got a new owner too if I'm not mistaken... Man, that would really suck!!

Anyway, thanks again, appreciate the tips!

I'm behind a CDN with special permissions Incase they attempt to ddos me. I also have security permissions that only allow my devices to login to the server they have to go through 3 authentications before the server accepts even a ping back

 

[bc12]

I'm behind a CDN with special permissions Incase they attempt to ddos me. I also have security permissions that only allow my devices to login to the server they have to go through 3 authentications before the server accepts even a ping back

Nice, that sounds like a pretty tight setup! Way beyond what I imagined. Some great ideas there for me to check out for sure.

Thanks for sharing that! I'm writing it down.

 

[abelcustoms]

Nice, that sounds like a pretty tight setup! Way beyond what I imagined. Some great ideas there for me to check out for sure.

Thanks for sharing that! I'm writing it down.

ssh keys aka secret keys are your way in most of the days you almost never want to be able to enter a password to gain access to your server as root. The server is set up to the point where I almost never have to ssh into it, everything can be managed and handled via a control panel.