LinuxCheatSheet

2018-02-12 03:01:26 [SSH][NOTIFICATION] Who is logging into your server?
Do you have some remote VPS and you want to control when someone login? Get notified in a telegram bot everytime a user enter ssh!
If you create a /etc/ssh/sshrc it will be executed at login before the shell start (refer to man ssh).
sshrc will run with the privilege of the user that has logged in, so be careful on what you write on it expecially if you login as root (but you don't, aren't you?). Following this small example https://github.com/gianluca-mascolo/ssh-telegram-notifier a curl to your telegram bot will be done at login.
Note: Don't blame me if you try it first on your ViPS (Very Important Production Server) and do something wrong and lock yourself out from ssh. In other words, try it before on a test server.

If you like it please share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G.

~How many stars do you rate this post?~ Aprire / Come

2018-02-09 03:10:04 [DNSMASQ][ADBLOCK] Block advertising on your whole network!

Dnsmasq is a local DNS caching server that can run on your computer. It's a popular software and some Linux distributions install it by default to cache DNS queries.
But it can do a lot more! For example it can be used to block advertising on your computer or even on your whole network. Let's start from this example configuration file:
~]# cat /tmp/dnsmasq.conf
server=208.67.222.222
interface=lo
listen-address=127.0.0.1
bind-interfaces
conf-file=/etc/adblock.conf
And /etc/adblock.conf contain the result of
curl -kL "https://is.gd/UadtgE" > /etc/adblock.conf
URL was shortened to fit on a mobile phone screen and it points to pgl.yoyo.org a popular ad blocking service. It contains entries like
address=/doubleclick.com/127.0.0.1
that Dnsmasq use to fake the resolution of doubleclick.com and other advertising domains redirecting them to 127.0.0.1.
Now start dnsmasq with systemctl restart dnsmasq if you are using a systemd based distro, or with dnsmasq -C /etc/dnsmasq.conf if you want to start it manually for test, and point your resolv.conf to
~]# cat /etc/resolv.conf
nameserver 127.0.0.1
Now try to load your favorite bloated web site full of advertising and voilà! You should now have cleaned it up without even using a browser plugins. And if you do it on a central DNS server in your network for example your openwrt router on another server distributed as an option by your DHCP you can even protect your whole network from advertising, including mobile phones attached to it! (Remember to change listen-address in dnsmasq.conf with your local network interface IP in that case).


If you like it please share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G.

~How many stars do you rate this post?~ Aprire / Come

2018-02-02 12:08:10 [NETCAT] Keep on hand a swiss army knife!
Netcat is a good old tool that you can use in many situations, from testing to prototyping a service. The name describe it well: it is a 'cat' over network. The simplest use you can do with it is to connect to a remote service and sending commands to it. For example, to make a HTTP request, open
nc -C www.google.com 80
and paste the lines
GET / HTTP/1.1
Host: www.google.com
Connection: close
This kind of use make nc a replacement for telnet to test some services (i.e. SMTP by sending HELO, RCPT TO and similar commands). The advantage of netcat over telnet is the ability to use UDP and other options like timeout control, the possibility to read input from another socket and more (see man nc). But the real power of netcat is that it can also listen on a port. Using it you can ever write a simple daemon with bash that can receive commands from a remote host and do some stuff. Look at this small example
If you run it on your pc (after installing netcat, of course), you can then open another shell and try to write to it.
]$ echo "test" | nc localhost 1234
]$ echo "q" | nc localhost 1234
Last but not least, please note that there are different flavours of netcat: gnu netcat, bsd netcat (the one used in this example) and nmap netcat (can do a lot of stuff!).

If you like it please vote the post and share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G.
(p.s. You can now assign a simple vote for the post, like with stars, from 3 to 1) Aprire / Come

2018-01-29 22:50:44 [PARALLEL] Do you have a big task to execute and you are short on time? Split it in smaller ones and try GNU Parallel

GNU parallel is available as package for the most common Linux distributions. Basically it works like xargs but the tasks it receives from standard input are executed in background and in parallel to maximize the use of all our cpu cores/threads.

Sometime there are tasks that simply take a lot of time if implemented with classical scripting, for example:

- you have a directory that contain a lot of subdirectories. i.e.
/srv/data/images/01
/srv/data/images/02
..
/srv/data/images/99
each subdir from 00 to 99 contain a lot of small files and you want to archive it in single subdir archives, like 00.tar.gz 01.tar.gz and so on.
You may try to implement it with a for cycle in bash, but you notice that it will take too much time. With GNU parallel you can do it faster with
seq -w 00 99 | parallel tar -C /srv/data/images/ -czf /srv/data/{}.tar.gz {}
You will obtain /srv/data/00.tar.gz /srv/data/01.tar.gz and so on.

- you have a big file in one server (let's say greater than 100GB). You want to distribute it to ten servers. You may scp to each server one by one, or do it with parallel. Suppose you have a text file with the ip address of each server
~]$ cat /tmp/server-list
10.0.0.1
10.0.0.2
10.0.0.3
10.0.0.4
...
and that you already have your ssh key in each server to login passwordless.
Then you can do it with
cat /tmp/server-list | parallel --progress rsync -aq /data/bigfile user@{}:/data/bigfile
rsync will be then spawned n times and you will copy the same file to multiple servers at one time.

If you like it please vote the post and share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G. Aprire / Come

2018-01-28 20:14:13 [SSH][PORTFORWARDING] Do you need to test an application behind a firewall? Do it via ssh!
SSH Forward local to remote
Suppose you have a remote server (target-server) behind some kind of firewall, and this server run a web app on port 80. Port 80 is not exposed to public Internet and for some reason you don't want (or you can't) change the firewall rules. Anyway, you can reach the target-server via ssh. From your computer, you can access the http application via ssh port-forwarding.
From your computer login to the target server.
[your@pc ~]$ ssh -C -L8080:localhost:80 remoteuser@target-server
After you login, leave the shell open
[remote@server ~]$ watch /bin/true
and iconify the window.
Then open a browser and point it to:
http://localhost:8080/

SSH Options used:
-C enable data compression
-L forward local to remote

So with the above command you forwarded local port 8080 to remote port 80 on localhost that is the target server itself. You can of course choose to forward on any other host reacheable by target server, if you do so, you are using target server as a bastion host.
Why the local port is 8080 and not 80? Two main reasons:
1. If you are not root on your pc, you can't open port below 1024
2. Port 80 may already be open on your pc

SSH Forward remote to local
You can also do the reverse: You have a testing webapp running on your pc and you want it to be reached by the target server:
[your@pc ~]$ ssh -C -R8080:localhost:80 remoteuser@target-server
Then after you login:
[remote@server ~]$ curl http://localhost:8080
The connections done on localhost 8080 to remote server will be forwarded to your pc on localhost 80.

SSH Dynamic Forwarding
This is, on my opinion, a very powerful feature.
From your pc connect to target server with
[your@pc ~]$ ssh -C -D8080 remoteuser@target-server
After you login, leave the shell open
[remote@server ~]$ watch /bin/true
and iconify the window.
Now open Firefox and configure the proxy. For the configuration choose "SOCKS5" proxy with SOCKS host localhost on port 8080.
In Firefox about:config change the value of key network.proxy.socks_remote_dns to true. Restart firefox. Now you can browse with Firefox as if you are browsing from the remote server, accessing all the hosts and applications that target server has right to access. Plus, if the target server has access to Internet, you can browse Internet as if you are in the remote location. Try it with http://ipecho.net/plain and you will see that your public IP is now the public IP of the remote server.

If you like it please vote the post and share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G. Aprire / Come

2018-01-27 01:01:47 [IPTABLES][CONNTRACK] Close the door!
Let's say you have a local network with server 192.168.1.1 and client 192.168.1.2, and normally they can ping each other.
—- server to client —-
[srv@telegram ~]# fping 192.168.1.2
192.168.1.2 is alive
[srv@telegram ~]#
—- client to server —-
[clt@telegram ~]# fping 192.168.1.1
192.168.1.1 is alive
[clt@telegram ~]#
This is your precious server and, for some reason, you want to prohibit the communication from client to server.
You may try with:
[srv@telegram ~]# iptables -A INPUT --src 192.168.1.2 -j DROP
If the client try to ping you now
[clt@telegram ~]# fping 192.168.1.1
192.168.1.1 is unreachable
But what is the effect on the server?
[srv@telegram ~]# fping 192.168.1.2
192.168.1.2 is unreachable
Why the server cannot reach the client if we close the Incoming traffic (INPUT) from client to server? You may note, that, in contrast with the previous post, you do not obtain a ping: sendmsg: Operation not permitted. What happen here is that the server send the ping to client, and the client receive it (unless, of course, it has some firewall rules too). But when the client reply to the server, the reply is dropped because of the firewall rule you inserted above. To close your server from communication received by client, but still allow it to communicate to client, you must use the connection tracking feature of NetFilter. You may do it with the module state of iptables. Flush your firewall rules and try to:
[srv@telegram ~]# iptables -F
[srv@telegram ~]# iptables -A INPUT -m state --src 192.168.1.2 --state NEW -j DROP
And now you have on the client
[clt@telegram ~]# fping 192.168.1.1
192.168.1.1 is unreachable
And on the server
[srv@telegram ~]# fping 192.168.1.2
192.168.1.2 is alive
We can interpret the second iptables rule used as: block incoming traffic from 192.168.1.2 if it is new traffic (that is: a packet never seen from the server kernel). In this case, when you initiate a ping from server to client, the kernel track the connection initiated from fping and can recognize the incoming packet from 192.168.1.2 (an echo reply) as related to the echo request you done.

iptables has a lot of modules that can help you in different task, have a look to man iptables-extensions for more informations.

If you like it please vote the post and share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G. Aprire / Come

2018-01-25 01:23:21 [IPTABLES] Deploy your first line of defence!
Firewall rules and packet management is a difficult skill to manage. Whether you are on cloud, on a physical server or a virtual machine you will soon or later need to secure your connection or reroute some packet to another destination. The Linux kernel do it through the Netfilter capability, and the standard tool to manipulate it is iptables. It is so called because Netfilter is divided in tables. Each table has a different purpose, and it is subdivided in chains. Each packet managed by the Linux Kernel cross one of this chains and is filtered/altered by a rule and a target. Let's see how it works starting with a simple example. As user root, try to
~]# ping -n 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8:
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss
~]# iptables -A OUTPUT --dst 8.8.8.8 -j DROP
~]# ping -n 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
ping: sendmsg: Operation not permitted
^C
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss
What's happened? You just manipulated the filter table: this is the default table used by iptables if you don't specify another one. This table is used to manipulate packets that:
- hits your server or Incoming Traffic.
- are generated from your server or Outgoing Traffic
- transits in your server of Forwarding Traffic
The filter table is hence divided in 3 chains: INPUT, OUTPUT and FORWARD.
You can view them with:
~]# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
DROP all -- 0.0.0.0/0 8.8.8.8
You can see in the output the three tables. Each table has a default policy, in this example: ACCEPT. That means that if no rule is specified in the chain, the packet crossing it is accepted. When you type:
iptables -A OUTPUT --dst 8.8.8.8 -j DROP
it is equivalent to:
iptables -t filter -A OUTPUT --dst 8.8.8.8 -j DROP that means:
In the filter table (-t), append (-A) as the last rule in the OUTPUT chain this rule: when destination (--dst) is 8.8.8.8 then direct it to target DROP. So, when you ping 8.8.8.8, your packets never exit your pc because they are dropped by Linux Netfilter.
If you want to discard the rule, you can flush the chain output with
~]# iptables -F OUTPUT
This command flush all of the rules you currently have in the output table. Be careful! If you use it on a remote production server and the OUTPUT table has the default policy DROP, you may cut it out completely from the Internet!

Wants to know more? Vote and share the post! Share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G. Aprire / Come

2018-01-21 13:33:18 [SSL][NMAP] How strong is your certificate?
Encryption of web communication has changed a lot, and while once it was an option (because it was CPU intensive), nowadays it is recommended to have it enable by default in your web site. To be effective, encryption must be configured properly.
But how to get out of acronym jungle out there? And how to check if your site is properly configured?
Let's start abandoning the word SSL in favor of TLS, that is Transport Layer Security. Then you can check the recommended cipher suites to use on mozilla website where you can find too example configurations for most common web servers and cloud providers. Usually the cipher choice is done balancing the security level you want to reach and the user base that will use your service (remember that not all people keep their software up to date and may have and old browser that does not support the latest cipher suite).
Then you want to check if your site is properly configured. There are some web services to do it, but why use some foreign service when you can simply use nmap on your computer? Try it with:
nmap -sV --script ssl-enum-ciphers -p 443 www.yoursite.com
It will take a while and it will return you a list of the currently configured ciphers and a vote (from A to F) of the strength of your site.

I hope you found this useful! Share the channel with your friends: http://t.me/linuxcheatsheet
Bye!
G. Aprire / Come

2018-01-18 16:13:56 [DATE][AWK] Working with date and time is a common task for everyone.
There are different tools to do it with Linux, most common one is date

~]$ date -d "2018/01/17 20:01:37" +%s
1516215697
~]$ date -d @1516215697 +"%Y/%m/%d %H:%M:%S"
2018/01/17 20:01:37
~]$

You can format date output as you need using + option followed by date format specifier (man date). The -d option allow you to see the date you specify, not the current system time. Please note that -d support relative dates, like
~]$ echo -n "Now: "; date
Now: Thu Jan 18 13:38:45 CET 2018
~]$ echo -n "Yesterday: "; date -d "1 day ago"
Yesterday: Wed Jan 17 13:39:07 CET 2018
~]$ echo -n "Future: "; date -d "+ 3 weeks"
Future: Thu Feb 8 13:39:38 CET 2018
~]$

This can be useful in different ways when scripting, usually when you want to obtain a time range relative to the current time to perform some action.
Another way to manipulate date and time is through AWK. Have a look to mktime and strftime.
~]$ echo | awk '{print mktime("2018 01 17 20 01 37")}'
1516215697
~]$ echo | awk '{print strftime("%Y/%m/%d %H:%M:%S",1516215697)}'
2018/01/17 20:01:37
~]$

Last but not least, you can even mix awk and date command. This is useful for example when you have a log with date in a certain format and you want to transform it to another one for any reason (i.e. sorting, or extracting a time range). This is the base syntax to use date inside awk:
~]$ echo | awk '{"date" | getline d; print d}'
Wed Jan 17 21:59:12 CET 2018
~]$

Have a look at this example of transformation of apache common log format.

I hope you like it! Please vote for the channel or share it
https://tchannels.me/c/linuxcheatsheet
http://www.telegramitalia.it/linuxcheatsheet/

Bye
G. Aprire / Come

2018-01-16 10:10:55 [XARGS] Do you need to repeat a command in from a set of results? Try xargs
This is one of the old school Unix commands, and it's still useful. Xargs reads by default from a pipe (stdin) a series of values, and use them as a parameter for the command you want to repeat. Let's say you have a bunch of docker images you want to remove from your laptop
[cheats@telegram ~]$ docker images -aq
29c1b56be99f
b802d38857fc
Instead of repeat docker rmi {image_id} many times, you combine the two commands this way
docker images -aq | xargs -r docker rmi
The -r option tell xargs to not execute commands if it receive no input (that is: you have no docker images at all to delete). xargs will automatically append every line in the output of docker images -aq as the last parameter of docker rmi, as if you typed docker rmi {image_id}.
If you want to execute a command and the parameter you want to pass is not the last one, use -I option of xargs, e.g.
command1 | xargs -I{} command2 {} otherparam.
See man xargs for more details.


I hope you like it! Please vote for the channel or share it
https://tchannels.me/c/linuxcheatsheet
http://www.telegramitalia.it/linuxcheatsheet/

Bye
G. Aprire / Come