Yii2 错误计入日志可查询方案学习

正文
参考资料

正文

为了更好的监控程序正常运行，我们需要把线上代码运行产生的错误都计入日志，以便查看是什么错误。yii2错误日志生成很方便，也有很多种方案。

可以先看下下面日志和错误处理两部分内容：

Yii 2.0 权威指南中请求处理（Handling Requests）: 日志（Logging）https://www.yiichina.com/doc/guide/2.0/runtime-logging

Yii 2.0 权威指南中请求处理（Handling Requests）: 错误处理（Handling Errors）https://www.yiichina.com/doc/guide/2.0/runtime-handling-errors

Yii2 异常及错误处理 https://ibaiyang.github.io/blog/yii2/2019/08/26/Yii2-异常及错误处理.html

接下来学习一种比较常用的技术手段。错误日志写入队列，Logstash再从队列把日志转入Elasticsearch，然后我们从Kibana可以实时查看日志记录。

生产者（Producer）项目

配置文件的components组件中写入errorHandler参数：

'errorHandler' => [
     'errorAction' => 'site/error-monitor',
],

在SiteConteoller中写入对应方法：

public function actionErrorMonitor()
{
    $exception = Yii::$app->errorHandler->exception;
    if ($exception instanceof \yii\web\HttpException) {
        $code = $exception->statusCode;
    } else {
        $code = $exception->getCode();
    }

    //404的错误不保存
    if ($code != 404) {
        ErrorService::AddError($exception, Yii::$app->request);
    }

    return $this->render('error');
}

common\services\ErrorService::AddError()方法处理错误详情，准备ElasticSearch数据：

public static function AddError($ex, $request)
{
    if ($ex && $ex->getMessage() != "Login Required") {
        $logs = [];
        $logs["indexname"] = "weberror";
        $logs["type"] = "adminWeb";
        if (Yii::$app->user->isGuest) {
            $logs["uid"] = 0;
            $logs["name"] = "0";
        } else {
            $logs["uid"] = Yii::$app->user->id;
            $logs["name"] = Yii::$app->user->identity->nickname;
        }

        $file = $ex->getFile();
        $line = $ex->getLine();

        $error_path = "file: {$file} [line: {$line}]";

        $logs["ip_address"] = $request->userIP;
        $logs["error_code"] = $ex->getCode();
        $logs["error_msg"] = $ex->getMessage();
        $logs["error_file"] = $error_path;
        $logs["error_url"] = $request->absoluteUrl;
        $logs["error_param"] = http_build_query($_POST);
        $logs["time_created"] = time();

        Queue::produceLogs($logs, 'logstash', 'app_logs_routing');
    }
}

其中用common\widgets\Queue::produceLogs()方法把日志加入队列：

public static function produceLogs($message, $exchange, $routing)
{
    try {
        $conn = new \AMQPConnection(Yii::$app->params['queue']);

        if (!$conn->connect()) {
            return false;
        }

        $message = json_encode($message);

        $channel = new \AMQPChannel($conn);
        $ex = new \AMQPExchange($channel);
        $ex->setName($exchange);
        $ex->setType(AMQP_EX_TYPE_DIRECT);
        $ex->setFlags(AMQP_DURABLE);

        $ex->declareExchange();

        if (!$ex->publish($message, $routing, 1)) {
            return false;
        }

        return true;

    } catch (\Throwable $ex) {
        return true;
    }
}

param文件中的队列配置：

'queue' => array (
    'host' => '10.10.110.121',
    'port' => '5672',
    'login' => 'mqweb',
    'password' => 'mq123456'
), 

日志内容加入了队列，接下来需要程序从队列服务器中读取日志，然后写入ES。

Logstash把RabbitMQ中的数据写入ES中，logstash配置，logstash.conf：

input {
    rabbitmq {
        exclusive => false
        host => '172.16.5.101'
        user => 'mquser'
        password => 'ABCabc123'
        vhost => '/'
        ack => false
        prefetct_count => 50
        auto_delete => flase
        durable => true
        exchange => 'exname'
        key => 'logs_routing'
        queue => 'logs'
        threads => 2
    }
}

output {
    elasticsearch {
        hosts => ["http://127.0.0.1:9200"]
        index => "%{indexname}-%{+YYYY.MM.dd}"
    }
}

在logstash目录下对logstash.conf配置文件执行运行：

./bin/logstash -f logstash.conf

一般错误日志记录，我们更多的是使用 kafka ， kafka相关内容，见：

https://ibaiyang.github.io/blog/yii2/2018/08/09/Yii2-kafka使用.html

查看数据

我们可以用kibana 查看 ElasticSearch 中的数据。看一下ES中写入的一条记录吧：

更多 ElasticSearch 和 kibana 相关的内容，见：

https://ibaiyang.github.io/blog/linux/2020/04/01/linux-安装Elasticsearch.html

消费者（Consumer）项目

上面隔了好长时间了，这里现在补上。除了上面的生产者，我们还需要一个消费者，并且在服务器上启动后长期运行。

例如运行命令：php rab log/start log

rab和yii一样，是入口文件，只不过改了名字。

rabbitMQ\controllers 文件夹下的LogController.php文件内容：

namespace rabbitMQ\controllers;

use yii;

class LogController extends BaseController
{
    //项目根目录
    private $root;
    
    //worker pidfile
    private $pidfile;
    
    //worker进程pid
    private $pids = [];

    //consumer
    private $consumer;
    
    //rabbitMQ全局配置
    private $setting = [];
    
    //worker对应queue,exchange,rooting绑定配置
    private $bindSetting = [];
    
    //worker实例
    private $worker;
    

    public function initConsumer($consumer)
    {
        $this->consumer = ucfirst($consumer);

        $this->root = Yii::$app->params['root'];
        
        $this->pidfile = Yii::$app->params['pidfile_root'] . $this->consumer;
        
        $this->bindSetting = Yii::$app->params['C' . $this->consumer];

        $this->setting = Yii::$app->params['logRabbitMQ'];

        $this->pids = file_exists($this->pidfile) ? 
            explode(',', file_get_contents($this->pidfile))
            : null;
    }

    /**
     * 启动consumer-worker
     */
    public function actionStart($consumer)
    {
        $this->initConsumer($consumer);

        $pid = pcntl_fork();

        if ($pid == -1) {
            die('could not fork');
        } else if ($pid) {
            exit(0);
        } else {
            posix_setsid();

            if (file_exists($this->pidfile))
            {
                file_put_contents($this->pidfile, ',' . posix_getpid(), FILE_APPEND);
            } else {
                file_put_contents($this->pidfile, posix_getpid());
            }

            $this->msgqueue();
        }
    }

    /**
     * 停止consumer-worker
     */
    public function actionStop($consumer)
    {
        $this->initConsumer($consumer);

        if (empty($this->pids))
        {
            echo "\n" . 'worker not start' . "\n";
            exit();
        }

        foreach ($this->pids as $pid)
        {
            posix_kill($pid, SIGTERM);
        }

        unlink($this->pidfile);

        echo "\n" . 'Stop Success' . "\n";
    }
    
    public function actionRestart($consumer)
    {
        $this->actionStop($consumer);
        sleep(2);
        $this->actionStart($consumer);
    }

    public function msgqueue()
    {
        $conn = new \AMQPConnection($this->setting);

        if (!$conn->connect())
        {
            echo "\n" . 'Connect Failed' . "\n";
            exit();
        }

        echo "\n" . 'Connect Success' . "\n";

        $channel = new \AMQPChannel($conn);
        
        $ex = new \AMQPExchange($channel);
        $ex->setName($this->bindSetting['exchange']);
        $ex->setType(AMQP_EX_TYPE_DIRECT);
        $ex->setFlags(AMQP_DURABLE);
        $ex->declareExchange();
        
        $q = new \AMQPQueue($channel);

        $q->setName($this->bindSetting['queue']);
        $q->setFlags(AMQP_DURABLE);
        $q->declareQueue();
        $q->bind(
            $this->bindSetting['exchange'],
            $this->bindSetting['routing']
        );

        if (!file_exists($this->root . '/workers/' . $this->consumer . '.php'))
        {
            echo "\n" . 'worker does not exist' . "\n";
            exit();
        }

        $workerName = 'rabbitMQ\\workers\\' . $this->consumer;

        $this->worker = new $workerName();

        $channel->qos(0,1);
        $q->consume(array($this->worker, 'run'));

        $conn->disconnect();
    }

}

配置文件rabbitMQ\config\params.php 节选：

'root' => dirname(__DIR__),
'pidfile_root' => '/data/log/rabbitMQ/',

'logRabbitMQ' => array (
    'host' => '10.10.110.120',
    'port' => '5672',
    'login' => 'mqweb',
    'password' => 'mq123456'
),

// 日志
'CLog' => array (
    'queue' => 'app_logs',
    'exchange' => 'logstash',
    'routing' => 'app_logs_routing'
),

进程文件rabbitMQ\workers\log.php 内容：

namespace rabbitMQ\workers;

use Yii;

class Log 
{
    public function run($envelope, $queue)
    {
        $msg = $envelope->getBody();
        $queue->ack($envelope->getDeliveryTag());
    }
}