分类目录归档:shell脚本

自己写的一个防止采集或CC或大量恶意访问的脚本

自己写的一个防止采集或CC或大量恶意访问的脚本

通过日志筛选ip

[root@xxxxx-web sh]# cat firewall_cron.sh
#!/bin/bash
#added by eugene@jjger.com —20240310

#定义脚本处理的ip文件,每天清理一次
firewall_cron_log=’/home/sh/firewall_cron.log’

#定义网站日志文件
logpath=’/data/log/nginx/access/’

arr_line=()
i=0

#while read line
#do
# arr_line[$i]=$line
# i=`expr $i + 1`
#done < $firewall_cron_log
#
#echo ${arr_line[*]}

while read line
do
if [ -z $ips ];then
ips=$line
else
ips=”$ips|$line”
fi
done < $firewall_cron_log

#echo $ips

#查找日志文件中的ip,$ips是已处理过的日志访问ip
if [ -z $ips ];then
max_visit=`grep -v ‘Googlebot’ $logpath/weblog.log | grep -v ‘baidu’ | grep -v ‘127.0.0.1’ |awk ‘{print $2}’ | sort -n | uniq -c | sort -n | tail -n 1`
else
max_visit=`grep -v ‘Googlebot’ $logpath/weblog.log | grep -v ‘baidu’ | grep -v ‘127.0.0.1’ | egrep -v $ips | awk ‘{print $2}’ | sort -n | uniq -c | sort -n | tail -n 1`
fi
#echo $?

#查看筛选出的ip,可注销下面这一行echo
echo $max_visit
#转化成数组后处理,默认处理5个,但后续更改为只处理一个ip,按计划任务轮训处理.
arr_max_visit=($max_visit)

if [ ${arr_max_visit[0]} -gt 2000 ]; then
iptables -L -n -v | grep ${arr_max_visit[1]}
if [ $? -eq 1 ];then
iptables -I INPUT -p tcp -s ${arr_max_visit[1]} -j DROP
echo ${arr_max_visit[1]} >> firewall_cron.log
fi
fi
exit
if [ ${arr_max_visit[2]} -gt 2000 ]; then
iptables -I INPUT -p tcp -s ${arr_max_visit[3]} -j DROP
fi
if [ ${arr_max_visit[4]} -gt 2000 ]; then
iptables -I INPUT -p tcp -s ${arr_max_visit[5]} -j DROP
fi
if [ ${arr_max_visit[6]} -gt 2000 ]; then
iptables -I INPUT -p tcp -s ${arr_max_visit[7]} -j DROP
fi
if [ ${arr_max_visit[8]} -gt 2000 ]; then
iptables -I INPUT -p tcp -s ${arr_max_visit[9]} -j DROP
fi

 

统计项目的代码量的shell脚本

#!/bin/bash
filesCount=0
linesCount=0
function funCount()
{
for file in ` ls $1 `
do
if [ -d $1″/”$file ];then
funCount $1″/”$file
else
declare -i fileLines
fileLines=`sed -n ‘$=’ $1″/”$file`
let linesCount=$linesCount+$fileLines
let filesCount=$filesCount+1
fi
done
}

if [ $# -gt 0 ];then
for m_dir in $@
do
funCount $m_dir
done
else
funCount “.”
fi
echo “filesCount = $filesCount”
echo “linesCount = $linesCount”

grep的贪婪和非贪婪匹配

grep -oE ‘/Public/v2/.*.png’ xxx.html

grep -oE ‘/Public/v2/.*\?.png’ xxx.html

无效,只能是贪婪模式

grep -oP ‘/Public/v2/.*?.png’ xxx.html

可以获取到所要的结果,不是贪婪模式,后面的?生效.

你正在寻找一种非贪婪(或懒惰)的比赛。 要在正则表达式中获得非贪婪匹配,您需要在量词后使用修饰符grep。 例如,您可以将grep -P更改为.*?

默认情况下,grep不支持非贪婪修饰符,但您可以使用grep -P来使用Perl语法。

mysql-5.7在centos7中按systemd启动,手册中说明

The following discussion covers these topics:

Overview of systemd

Configuring MySQL Using systemd

Configuring Multiple MySQL Instances Using systemd

Migrating from mysqld_safe to systemd

Overview of systemd

systemd provides automatic server startup and shutdown. It also enables manual server management using the systemctl command. For example:

systemctl {start|stop|restart|status} mysqld
Alternatively, use the service command (with the arguments reversed), which is compatible with System V systems:

service mysqld {start|stop|restart|status}
For the systemctl or service commands, if the MySQL service name is not mysqld, use the appropriate name (for example, mysql on SLES systems).

Support for systemd includes these files:

mysqld.service: systemd service unit configuration, with details about the mysqld service.

mysqld@.service: Like mysqld.service, but used for managing multiple MySQL instances.

mysqld.tmpfiles.d: File containing information to support the tmpfiles feature. This file is installed under the name mysql.conf.

mysqld_pre_systemd: Support script for the unit file. This script assists in creating the error log file only if its location matches the pattern /var/log/mysql*.log. In other cases, the error log directory must be writable or the error log must be present and writable for the user running the mysqld process.

On platforms for which systemd support is installed, scripts such as mysqld_safe and the System V initialization script are not installed because they are unnecessary. For example, mysqld_safe can handle server restarts, but systemd provides the same capability, and does so in a manner consistent with management of other services rather than using an application-specific program.

As of MySQL 5.7.13, on platforms for which systemd support is installed, systemd has the capability of managing multiple MySQL instances. For details, see Configuring Multiple MySQL Instances Using systemd. Consequently, mysqld_multi and mysqld_multi.server are not installed because they are unnecessary.

Configuring MySQL Using systemd

To add or change systemd options for MySQL, these methods are available:

Use a localized systemd configuration file.

Arrange for systemd to set environment variables for the MySQL server process.

Set the MYSQLD_OPTS systemd variable.

To use a localized systemd configuration file, create the /etc/systemd/system/mysqld.service.d directory if it does not exist. In that directory, create a file that contains a [Service] section listing the desired settings. For example:

[Service]
LimitNOFILE=max_open_files
PIDFile=/path/to/pid/file
Nice=nice_level
LimitCore=core_file_limit
Environment=”LD_PRELOAD=/path/to/malloc/library”
Environment=”TZ=time_zone_setting”
The discussion here uses override.conf as the name of this file. Newer versions of systemd support the following command, which opens an editor and permits you to edit the file:

systemctl edit mysqld
Whenever you create or change override.conf, reload the systemd configuration, then tell systemd to restart the MySQL service:

systemctl daemon-reload
systemctl restart mysqld
Support for configuration using override.conf was added in MySQL 5.7.7.

With systemd, the override.conf configuration method must be used for certain parameters, rather than settings in a [mysqld_safe] or [mysqld] group in a MySQL option file:

For some parameters, override.conf must be used because systemd itself must know their values and it cannot read MySQL option files to get them.

Parameters that specify values otherwise settable only using options known to mysqld_safe must be specified using systemd because there is no corresponding mysqld parameter.

For additional information about using systemd rather than mysqld_safe, see Migrating from mysqld_safe to systemd.

You can set the following parameters in override.conf:

To specify the process ID file:

As of MySQL 5.7.10: Use override.conf and change both PIDFile and ExecStart to name the PID file path name. Any setting of the process ID file in MySQL option files will be ignored.

Before MySQL 5.7.10: Use PIDFile in override.conf rather than the –pid-file option for mysqld_safe or mysqld. systemd must know the PID file location so that it can restart or stop the server. If the PID file value is specified in a MySQL option file, the value must match the PIDFile value or MySQL startup may fail.

To set the number of file descriptors available to the MySQL server, use LimitNOFILE in override.conf rather than the –open-files-limit option for mysqld_safe or mysqld.

To set the maximum core file size, use LimitCore in override.conf rather than the –core-file-size option for mysqld_safe.

To set the scheduling priority for the MySQL server, use Nice in override.conf rather than the –nice option for mysqld_safe.

Some MySQL parameters are configured using environment variables:

LD_PRELOAD: Set this variable if the MySQL server should use a specific memory-allocation library.

TZ: Set this variable to specify the default time zone for the server.

There are multiple ways to specify the value of environment values that should be in effect for the MySQL server process managed by systemd:

Use Environment lines in the override.conf file. For the syntax, see the example in the preceding discussion that describes how to use this file.

Specify the values in the /etc/sysconfig/mysql file (create the file if it does not exist). Assign values using the following syntax:

LD_PRELOAD=/path/to/malloc/library
TZ=time_zone_setting
After modifying /etc/sysconfig/mysql, restart the server to make the changes effective:

systemctl restart mysqld
To specify options for mysqld without modifying systemd configuration files directly, set or unset the MYSQLD_OPTS systemd variable. For example:

systemctl set-environment MYSQLD_OPTS=”–general_log=1″
systemctl unset-environment MYSQLD_OPTS
After modifying the systemd environment, restart the server to make the changes effective:

systemctl restart mysqld
MYSQLD_OPTS can also be set in the /etc/sysconfig/mysql file.

Configuring Multiple MySQL Instances Using systemd

As of MySQL 5.7.13, on platforms for which systemd support is installed, systemd has the capability of managing multiple MySQL instances. Consequently, mysqld_multi and mysqld_multi.server are not installed because they are unnecessary.

To use multiple-instance capability, modify my.cnf to include configuration of key options for each instance. For example, to manage two instances named replica01 and replica02, add something like this to the file:

[mysqld@replica01]
datadir=/var/lib/mysql-replica01
socket=/var/lib/mysql-replica01/mysql.sock
port=3307
log-error=/var/log/mysqld-replica01.log

[mysqld@replica02]
datadir=/var/lib/mysql-replica02
socket=/var/lib/mysql-replica02/mysql.sock
port=3308
log-error=/var/log/mysqld-replica02.log
The replica names shown here use @ as the delimiter because that is the only delimiter supported by systemd.

Instances then are managed by normal systemd commands, such as:

systemctl start mysqld@replica01
systemctl start mysqld@replica02
To enable instances to run at boot time, do this:

systemctl enable mysqld@replica01
systemctl enable mysqld@replica02
Use of wildcards is also supported. For example, this command displays the status of all replica instances:

systemctl status ‘mysqld@replica*’
For management of multiple MySQL instances on the same machine, systemd automatically uses a different unit file (mysqld@.service rather than mysqld.service). In that unit file, %I and %i reference the parameter passed in after the @ marker and are used to manage the specific instance. For a command such as this:

systemctl start mysqld@mysql1
systemd starts the server using a command such as this:

mysqld –defaults-group-suffix=@%I …
The result is that the [server], [mysqld], and [mysqld@mysql1] option groups are read and used for that instance of the service.

Migrating from mysqld_safe to systemd

Because mysqld_safe is not installed when systemd is used, options previously specified for that program (for example, in an [mysqld_safe] option group) must be specified another way:

Some mysqld_safe options are also understood by mysqld and can be moved from the [mysqld_safe] option group to the [mysqld] group. This does not include –pid-file or –open-files-limit. To specify those options, use the override.conf systemd file, described previously.

For some mysqld_safe options, there are similar mysqld options. For example, the mysqld_safe option for enabling syslog logging is –syslog. For mysqld, enable the log_syslog system variable instead. For details, see Section 6.4.2, “The Error Log”.

mysqld_safe options not understood by mysqld can be specified in override.conf or environment variables. For example, with mysqld_safe, if the server should use a specific memory allocation library, this is specified using the –malloc-lib option. For installations that manage the server with systemd, arrange to set the LD_PRELOAD environment variable instead, as described previously.

shell的登陆交互

#!/bin/bash
# test.sh user pw
if [ ! $# -eq 2 ] ; then
echo “请输入用户名和密码以空格分开!”
exit

else
name=”$1″
passwd=”$2″
fi

cat hosts | while read hosts
do

echo “正在$hosts上用户$name”
expect <” {
send “enable\r” ;exp_continue
}
}
EOF
echo “成功建立”
done

sed单行脚本处理文本

————————————————————————-
SED单行脚本快速参考(Unix 流编辑器) 2005年12月29日

在以下地址可找到本文档的最新(英文)版本:
http://sed.sourceforge.net/sed1line.txt
http://www.pement.org/sed/sed1line.txt

其他语言版本:
中文 – http://sed.sourceforge.net/sed1line_zh-CN.html
捷克语 – http://sed.sourceforge.net/sed1line_cz.html
荷语 – http://sed.sourceforge.net/sed1line_nl.html
法语 – http://sed.sourceforge.net/sed1line_fr.html
德语 – http://sed.sourceforge.net/sed1line_de.html

葡语 – http://sed.sourceforge.net/sed1line_pt-BR.html

继续阅读