From 4f13e9d3b0a5bf2053188f3b78971e956afa7518 Mon Sep 17 00:00:00 2001 From: Harold-Ran <56714856+Harold-Ran@users.noreply.github.com> Date: Sat, 4 Dec 2021 20:09:55 +0800 Subject: [PATCH] Add files via upload --- Task3/Task3 模型建立之CNN+LSTM.ipynb | 2553 ++++++++++++++++++++++++++ 1 file changed, 2553 insertions(+) create mode 100644 Task3/Task3 模型建立之CNN+LSTM.ipynb diff --git a/Task3/Task3 模型建立之CNN+LSTM.ipynb b/Task3/Task3 模型建立之CNN+LSTM.ipynb new file mode 100644 index 0000000..ad9981e --- /dev/null +++ b/Task3/Task3 模型建立之CNN+LSTM.ipynb @@ -0,0 +1,2553 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.043894, + "end_time": "2021-11-10T09:14:57.523940", + "exception": false, + "start_time": "2021-11-10T09:14:57.480046", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "# Datawhale 气象海洋预测-Task3 模型建立之 CNN+LSTM\n", + "本次任务我们将学习来自TOP选手“学习AI的打工人”的建模方案，该方案中采用的模型是CNN+LSTM。\n", + "\n", + "在本赛题中，我们构造的模型需要完成两个任务，挖掘空间信息以及挖掘时间信息。那么，说到挖掘空间信息的模型，我们会很自然的想到CNN，同样的，挖掘时间信息的模型我们会很容易想到LSTM，我们本次学习的这个TOP方案正是构造了CNN+LSTM的串行结构。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.04014, + "end_time": "2021-11-10T09:14:57.605087", + "exception": false, + "start_time": "2021-11-10T09:14:57.564947", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## 学习目标\n", + "1. 学习TOP方案的数据处理方法。\n", + "2. 学习TOP方案的模型构建方法。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.040277, + "end_time": "2021-11-10T09:14:57.685242", + "exception": false, + "start_time": "2021-11-10T09:14:57.644965", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## 内容介绍\n", + "1. 数据处理\n", + " - 增加月特征\n", + " - 数据扁平化\n", + " - 空值填充\n", + " - 构造数据集\n", + "2. 模型构建\n", + " - 构造评估函数\n", + " - 模型构造与训练\n", + " - 模型评估\n", + "3. 总结" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.041248, + "end_time": "2021-11-10T09:14:57.766859", + "exception": false, + "start_time": "2021-11-10T09:14:57.725611", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## 代码示例" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.041028, + "end_time": "2021-11-10T09:14:57.848372", + "exception": false, + "start_time": "2021-11-10T09:14:57.807344", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "### 数据处理" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.041407, + "end_time": "2021-11-10T09:14:57.930553", + "exception": false, + "start_time": "2021-11-10T09:14:57.889146", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "该TOP方案的数据处理主要包括四部分：\n", + "\n", + "1. 增加月特征。将序列数据的起始月份作为新的特征。\n", + "2. 数据扁平化。将序列数据按月拼接起来通过滑窗增加数据量。\n", + "3. 空值填充。\n", + "4. 构造数据集。随机采样构造数据集。" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:14:58.216163Z", + "iopub.status.busy": "2021-11-10T09:14:58.215195Z", + "iopub.status.idle": "2021-11-10T09:15:03.590431Z", + "shell.execute_reply": "2021-11-10T09:15:03.589683Z", + "shell.execute_reply.started": "2021-11-10T08:55:36.925247Z" + }, + "papermill": { + "duration": 5.499137, + "end_time": "2021-11-10T09:15:03.590597", + "exception": false, + "start_time": "2021-11-10T09:14:58.091460", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "import netCDF4 as nc\n", + "import random\n", + "import os\n", + "from tqdm import tqdm\n", + "import math\n", + "import pandas as pd\n", + "import numpy as np\n", + "\n", + "import seaborn as sns\n", + "color = sns.color_palette()\n", + "sns.set_style('darkgrid')\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline\n", + "\n", + "import torch\n", + "from torch import nn, optim\n", + "import torch.nn.functional as F\n", + "from torch.utils.data import Dataset, DataLoader\n", + "\n", + "from sklearn.metrics import mean_squared_error" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:03.682821Z", + "iopub.status.busy": "2021-11-10T09:15:03.682060Z", + "iopub.status.idle": "2021-11-10T09:15:03.688267Z", + "shell.execute_reply": "2021-11-10T09:15:03.688778Z", + "shell.execute_reply.started": "2021-11-10T08:24:19.206477Z" + }, + "papermill": { + "duration": 0.055671, + "end_time": "2021-11-10T09:15:03.688955", + "exception": false, + "start_time": "2021-11-10T09:15:03.633284", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 固定随机种子\n", + "SEED = 22\n", + "\n", + "def seed_everything(seed=42):\n", + " random.seed(seed)\n", + " os.environ['PYTHONHASHSEED'] = str(seed)\n", + " np.random.seed(seed)\n", + " torch.manual_seed(seed)\n", + " torch.cuda.manual_seed(seed)\n", + " torch.backends.cudnn.deterministic = True\n", + " \n", + "seed_everything(SEED)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:03.824679Z", + "iopub.status.busy": "2021-11-10T09:15:03.824122Z", + "iopub.status.idle": "2021-11-10T09:15:03.828366Z", + "shell.execute_reply": "2021-11-10T09:15:03.828769Z", + "shell.execute_reply.started": "2021-11-10T08:24:19.220289Z" + }, + "papermill": { + "duration": 0.096679, + "end_time": "2021-11-10T09:15:03.828935", + "exception": false, + "start_time": "2021-11-10T09:15:03.732256", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "CUDA is available! Training on GPU ...\n" + ] + } + ], + "source": [ + "# 查看CUDA是否可用\n", + "train_on_gpu = torch.cuda.is_available()\n", + "\n", + "if not train_on_gpu:\n", + " print('CUDA is not available. Training on CPU ...')\n", + "else:\n", + " print('CUDA is available! Training on GPU ...')" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:04.042120Z", + "iopub.status.busy": "2021-11-10T09:15:04.041526Z", + "iopub.status.idle": "2021-11-10T09:15:04.265472Z", + "shell.execute_reply": "2021-11-10T09:15:04.265008Z", + "shell.execute_reply.started": "2021-10-21T07:25:02.230095Z" + }, + "papermill": { + "duration": 0.395064, + "end_time": "2021-11-10T09:15:04.265600", + "exception": false, + "start_time": "2021-11-10T09:15:03.870536", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 读取数据\n", + "\n", + "# 存放数据的路径\n", + "path = '/kaggle/input/ninoprediction/'\n", + "soda_train = nc.Dataset(path + 'SODA_train.nc')\n", + "soda_label = nc.Dataset(path + 'SODA_label.nc')\n", + "cmip_train = nc.Dataset(path + 'CMIP_train.nc')\n", + "cmip_label = nc.Dataset(path + 'CMIP_label.nc')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.040337, + "end_time": "2021-11-10T09:15:04.346658", + "exception": false, + "start_time": "2021-11-10T09:15:04.306321", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 增加月特征" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.040016, + "end_time": "2021-11-10T09:15:04.427539", + "exception": false, + "start_time": "2021-11-10T09:15:04.387523", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "本赛题的线上测试集是任意选取某个月为起始的长度为12的序列，因此该方案中增加了起始月份作为新的特征。但是使用整数1~12不能反映12月与1月相邻这一特点，因此需要借助三角函数的周期性，同时考虑到单独使用sin函数或cos函数会存在某些月份的函数值相同的现象，因此同时使用sin函数和cos函数作为两个新增月份特征，保证每个起始月份的这两个特征组合都是独一无二的，并且又能够很好地表现出月份的周期性特征。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.040553, + "end_time": "2021-11-10T09:15:04.508382", + "exception": false, + "start_time": "2021-11-10T09:15:04.467829", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "我们可以通过可视化直观地感受下每个月份所构造的月份特征组合。" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:04.603365Z", + "iopub.status.busy": "2021-11-10T09:15:04.602559Z", + "iopub.status.idle": "2021-11-10T09:15:05.149292Z", + "shell.execute_reply": "2021-11-10T09:15:05.148810Z", + "shell.execute_reply.started": "2021-10-21T07:25:02.336139Z" + }, + "papermill": { + "duration": 0.599664, + "end_time": "2021-11-10T09:15:05.149428", + "exception": false, + "start_time": "2021-11-10T09:15:04.549764", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAABJAAAAEvCAYAAAAadDsuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACnMElEQVR4nOzdd3wUdf7H8ddsy2Y3PQRCCZBQhk4oKqgoKiAKglQBey+np9797mzneWe509M7rKen2JXeBQtYUFFUemdpoRNKkk3Zzfb5/ZGAqLCbQJJJdj/Px8MHye5M8vaTKbufnfl+FU3TEEIIIYQQQgghhBDiVAx6BxBCCCGEEEIIIYQQ9Zs0kIQQQgghhBBCCCFEWNJAEkIIIYQQQgghhBBhSQNJCCGEEEIIIYQQQoQlDSQhhBBCCCGEEEIIEZY0kIQQQgghhBBCCCFEWCa9A5yOUCikBYOa3jFEHTAaFeRvHZ7UKDypT2RSo/CkPpFJjcKT+kQmNQpP6hOZ1Cg8qU9kUiMhKpjNxqNAxsmea5ANpGBQw+l06x1D1IGUFJv8rSOQGoUn9YlMahSe1CcyqVF4Up/IpEbhSX0ikxqFJ/WJTGokRIWMjMTdp3pObmETQgghhBBCCCGEEGFJA0kIIYQQQgghhBBChCUNJFFvKcov/xVC1DzZz4QQQgghhBBV0SDHQBLRS1HAajUTbwpitNoBaNQokaDHRXnAiMfjR5Ox7YQ4I8f2M8VkxG41AxX7mcvjRwsEZT8TQgghhBBC/IY0kES9YTYbSU4wQeFOlGUvwZaF4C2BuCSMHYZg73sP9rQcissC+P1BveMK0SCZzUZsCVZ2Fbh441sHizcdotQTINFqYmCnJtzaL4fW6XbcZR7Zz4QQQgghhBDHSQNJ1Atms5HkRDPKgvtg9Qe/fLK8CFZ/gLL6A+hxLclDJ1Jciry5FaKazGYj9kQrD89Zz4wV+37xnNPtZ8aKfcxYsY+xvbN4akQXXKXSRBJCCCGEEEJUkDGQhO4UBZITTCdvHv3a6vdRFtxfsbyM2SJElSkK2BJO3jz6tekr9vLInA3YEqyynwkhhBBCCCGAGroCSVXVt4ChwGGHw9HlJM8rwAvA5YAbuMHhcKyqfO564C+Viz7pcDjerYlMouGwWs1QsCNy8+iY1e9D37uJi2+Nx+Ov3XBCRAmr1cyuo66IzaNjpq/Yyy39smlqM8t+JoQQQgghhKixK5DeAQaHef4yoF3lf7cBrwKoqpoGPAacA5wNPKaqamoNZRINRLwpiPLDy9VaR1n2Mjaz3FojRFUpJiNvLN1ZrXUmfZuHwWyspURCCCGEEEKIhqRGrkByOBzfqKraOswiw4H3HA6HBvygqmqKqqpNgf7AYofDUQigqupiKhpRU2oil6ingj4UXxmKrwSDvwyjKwgb51bvZ2ycg/GsmzG6DYTMSWiWBDDGyVzkQgCaplHuD1HmDeDyBXH7AxidHj5ed7BaP+fj9Qe46fzWBNxebGYTNosRo0H2MSGEEEKIqKCFULwlFe/LvMUofhcEvChBHwQr/lWCPkBDUwygGCveb1V+rZni0cw2NLP95/8sdjRLkrwvi1J1NYh2c2DvCd/vq3zsVI+HZTQqpKTYajSgqAEBLxTvRXHuQnHuhbJ8cB1GKTsMrkMV/7qPogQ8Z/67fGXwen/STnhIM1rAlo5mbwL2DEhoXPF1QhO0lJZoKS0hpRWYo2vbMRoNsj+EEU310TQNZ7mfvYXl7Ctyc6DYw9EyL0dKvRwt81V8XealuNxPSDvz31fmDTL4+W9/8Zg9zkhGQhyNEuIq/k20kJEQR5MkK1mp8bRItdEkMQ5DFDWaomkbqi1So/CkPpFJjcKT+kQmNQpP6hNZVNUoFICS/ShFu6D0IEpZPpTmo5QdrPjedQQ8TvCUoFADLxp/RTOYwZZe+d6sEdgaodkbQ3ILtKQstOQsSM6C+FRpNDUwDXIWtmBQw+l06x0jNmkahrIDmAodGAu3YSzcirF4F8bSPRjK8n9xANJQ0OLTCdkyCNkaE2qaQyg+HS0umZAlEc2SCHEJJGU0gakTwFta9RyWBBg1idKjh8BbhuIrxeArQXEXYHAfxlCSjyF/HQb3URTtl7e6heIzCCa3JJjUikBae4Kp7QiktSeU1BIMDe92nZQUm+wPYTTE+nj8QXYXlrOz0MXOo252FbrZX+zhQLEHl++X23OcyUC63UK6zUKzpDi6Nk0kOd5MgsWIPc5EgsVIQpyJZo0TufGd5ZR5AlXOYY8z8uzo7hw8WlZxNZM3SLHHT6HbT4HLx6YDxRS4fZR5f5nJbFRommSlWbKVVqnx5DSyk5NmI6eRjSSruUZqVJca4jZU16RG4Ul9IpMahSf1iUxqFJ7UJ7KGWCOlvBBTwWaMhQ6MzryK92XFuzCW7kMJ/XIMy5AliZA9k5C9CaHGPQnFpaDFJVe8N4tLRotLQjMnVHwob7RU/htX8a9iAC0EWhBFC4GmQSiAEvSg+F0ofnflvy4UXxmG8kKU8qMYygsxuI9iKNiFwX0YJVD+y0xmO6GklgRS2xKs/C+Q2o5gSjaY4uuylOIEGRmJp3yurhpI+4GsE75vUfnYfipuYzvx8SV1lElEEvRjKnRgOrwW0+F1mAo2YSzchsFfdnyRUHwjAilt8Dc/j2BSS4JJLQklZRFMyiJkawyGyJtYMNGAsdPwqg+iDdB5BMFWF+BJC4VfTguhuI9iLN2LsWQvxpI9GEr2YCzZi/nAMqxbZ/+8qDGu4oCV3gF/424EGncn0KiTHLxErSpy+9h8qIzNh0rZcqiMHUdd7C/2HL+CyGhQyEqx0iIlnp4tkmmWbKV5ckVzpmmSFbvFiFKFT25siVYu65JZ5UG0AYZ0bcaF7RrhzkwIu5zHH+RwmY/9xeUcqGx07S/2sN/pYe3+Ysr9P++njewWctJtqI0T6JSZSMfMBJolWav0/yCEEEIIEZO0EEZnHqbDqzEd2YSpcAvGgi0Y3YePLxIyJxBMbk2gUWd8bYYQTG5V8f4soRkhexP978LQNBRPEcbSfRhK92Es3V/xb/EuzIfXEbd9wfGLETQUginZBBp1IZDRhUBGVwIZXdCsMlyy3uqqgTQfuFtV1alUDJhd7HA4Dqqq+hnwjxMGzh4EPFRHmcSvGEr3Yz7wI+ZDKzEdWoupYDNK0AtAKC6ZQKPOeDuMJpCmEkxrRyBNrZGduDxgxN73HpRqNJC0vvfg9huBCA0kxYBmb0zA3phAZq/fPu0rxVi4FVPl1VSmIgeWPUuwOmZW/B7FSDBNrWgoZfbE36wPweRsudRSnBZ/MMTmQ2Ws3V/MhoOlbD5UysES7/HnW6bG075xAoM7NiY73U5Ouo2WqfGYjWc+34EWCHJrv5xqNZBu7ZdNyB95sHqr2UjL1Hhapv622RrSNPJLvOwscJFX4GZHgZudR11MWbWfQGWXLNlqomOTRDplJtC9eTLdmiWRENcgL5AVQgghhDhjSnkB5vyVmA6twXxoNabDazH4SoDKD73T2uNv2Z/y9A4E0jsQSOuAZsuo3+9RFAUtPo1AfBo07vbb5wMejMV5mAq3YyzahqlgE+ZDq7Bun398kWBiC/xNehDI7I2/6VkVH/ZX4YIFUXNqpNqqqk6h4kqiRqqq7qNiZjUzgMPheA34GLgc2A64gRsrnytUVfUJYHnlj3r82IDaopZpGkbnTswHfqhoGh38CWNpxRvLkNlOIKMr5V1vINC4O/7G3Qgltaq1A5LH48eelgM9rqnaVUg9roW0bLxOb+RlI9AsiQQye/2yuaRpGFwHK666OrwO85G1xO38lPjNU4GKW+D8zc7G3/RsfM36EGzUseKyTiF+xeMPsu5ACWv2F7N6XzHrD5biDVQ0PZsnW+mcmcSY3IorcdTGCbXaNPF4/LROtzOmd4sqNZHG9s6iVbqdEqfrjH6vQVFoVnnF1Pk56ccf9wVC7ChwsTm/lE2HyticX8q7P+0lqO3FoED7jARyWyTTo3kSPVokk2qznFEOIYQQQoj6SnEfxXzgBywHlmHe/wOmQgdQ8WF2IL0D3nbDCDTOxd8kl2BquwY57EZEJivB9I4E0zv+4mHFU4TpyIbK/9Zhzl+BdftHAGgmG/4mPfA3PQt/1vn4m/QEo7xmrE2KptX8oFm1ze8Pag3t/tT6QPEUYdm7FPPeJVj2foOxrGJGplB8I/zNzsHf9Gz8zc4hkN6xzg9KZrOR5EQzyoL7YfX7p16wx7VoQydSXOrHX4UrI2qMpmF07qhotv264WZNxZd1Ab6sC/G3vICQPbPuctEw79euS3VZH03T2HHUzbJdhfywq4jV+4vxBzUMCrTLSKBHZUOke/Nk0u11f3Izm43YE608MmcD01fsPeVyY3tn8dSILrhKPXW6n5X7g6w/RcNNbZxAn9ap9GmVSvfmSTVyVVZVyT4WmdQoPKlPZFKj8KQ+kUmNwpP6RFanNQp4KhpGu7/Csm/pzw0jkw1/07PwNe9LoOlZ+DO6gVmG0/g1Q+l+zAeXY85fjungioo7Z7QQmsmGr3kf/C364cs6n2Bah/p9VVY9lZGRuBLofbLnpIEUzTQN05H1WPIWYdmzBNPhtShohOKS8bc4D1/WBfibn1tvbskym40kJ5igcCfKspdhywLwlkBcEnQYitb3HkjLprgsULfNo1OouOVvGZa932LZ8w2G8iMABNJUfC3748seiD/zrFpvxskLgvBquz5l3gDf5xWybFcRP+4u4kiZD4CcdBt9WqdydsuKhkd9uSXLbDZiS7Cyq8DFpG/zWLQpn1JPgESriUGdMrm1Xzat0u24y+q2eXQyx275W7HHyQ+7i1h3oIRgSCPebKBXVgp9W6dxYdt0miTG1WoO2ccikxqFJ/WJTGoUntQnMqlReFKfyGq7Robi3Vj2fFXRNNr/HUrAg2aMw9+sD77mffE370sgoxsYG96EI3pTvMWY9y/Dsu9bzHuXYnLuACBoa4yv9QB82YPwtThPxratImkgxZKgH/PBn7Ds/JS4vM8wlh1AUwwEmvSsuEqm5YUEGnevt/eKKgrExZmxmYMYrfbjjwc9Ltx+I16vn3q5yWoaxoLNWPZ8jWXvN5gP/IgS8hGypuFtPRBfzqX4svrVykFLXhCEVxv1KXD5+HpHAV9tO8qKPU4CIY0kq4mzW6bSt3Uq57ROrfWmxpk4tp8ZzEbsJ8yK5vL4CfmD9XY/c/kCrNhTzI+7i/hhVyF7nR4AOmUm0r9tOhe2TSc7zVbjA3LLPhaZ1Cg8qU9kUqPwpD6RSY3Ck/pEVuM10jRMRzdi2fExcTs/xlS0HYBgUit8rS7C1/IifM3PlSuMaoGh9ADmfd9WNOv2LMHgL0MzWfFlXYiv9UC8rS+pGDNKnJQ0kKJd0IdlzxLidizEsutzDN7i4zuIN2cwvlaXoMWn6Z2y2hQFkpNtFBe76+Wb2XAUXxnmPUuIy/sMy64vMPhK0Ezx+Fr2x5tzGd7sS8Fij/yDqkBeEIRXU/U5UOzhy21HWbLtKOsOlKABLVKs9G/biP5t0+nSNAmjQf8r+aqrIe9nuwrcfL2jgCXbj7LhYClQMRD5hW3SGdghgw6NE2qkmST7WGRSo/CkPpFJjcKT+kQmNQpP6hNZjdRI0zAdXkPcjoXE7fgEY8luNMWAv1lffNkD8bW6uN7c/REzgj7MB34gLm8RlrzFGMv2o6Hgb3o23nbD8LYZgmZrpHfKekUaSNFIC1XsCFvnErdjIQZvMaG4FHzZA/FmX4ov68Ko6GZHxcku6MN84Efidn6KJe9TjK5DaKZ4vNmD8LYfgS/rgtMe7K0hv/mvC2daH2e5n88dR/h082HWHqiY+aJ9hp3+7RpxUdtGtGlU81e76CEa9rMjZV6+2VHAku0Fx68Ka5kaz+AOjbm0Y+OTzhBXFbKPVU00bEO1SeoTmdQoPKlPZFKj8KQ+4Z3p+d5YsBmrYxZx2z6qaFAYTPhbnI+3zeV4sy9Fi0+P/ENE7au8ayQu7zPitn2EqWhrRYOvxfl42w7DmzMYzZqid0rdSQMpWlReBhm3dQ5x2+ZhdOWjmWx4cwbjbX8lvhb9ou6e2ag72WkhTAdXYN02l7jtH2HwFBGKS8Hbdije9lfib3p2xBndFAWsVjPxpt/e5lceMOLx1M/bj+rKsfoopt/enqUFghHr4/EH+WZHAZ9uPsz3u4oIhjSy021c1rExA9UMWqQ0/Mbsr0Xbflbi8fPl1qN8tuUwK/cWo1Fxm9ulHTIYpGbQKCH87YVnug3Fomjbhmqa1CcyqVF4Up/IpEbhSX1+60xfUxtch4jbOherYxamgk1oBlPFHSBtr8DXeoA0IhoAY8EW4rbNx7ptXsXVYgYzvpYX4ek4Bl+rS2J2RjdpIDVwisdJ3La5WDdNwXx04/EN29v+SrytB0bFlUanEtUnu6Afy95vKhqCeZ+hBMoJJrWivNN4vB1Gn3Q2t18ONP4SbFl4wkDjQyoHGs+pNwON17UTB4h+49udLN506PgA0QM7NeHWfjm0PskA0ZqmselQGXPXHWSx4wguX5CMBAuXdmjM4I6NaZ9hj4orjU4lmvezw6VeFldeRbblcBlGBc7LSWd410zOzU7D9KvbDk93G4p10bwN1QSpT2RSo/CkPpFJjcKT+vzSab+mDniI2/kJVsdMzHu/RdFC+Bvn4lFH4W03TK40aqg0DdORdcRtm0/c1rkY3YcIxafjaT8ST8erCKZ30DthnZIGUkOkaZgP/IB10xTidixECXrxZ3TF03FcxcHJmqp3wjoRMyc7v7viZLR5Kpb9y9AUI75Wl+DpNB5fq4vAYKo40SWaURbcB6s/OPXP6nEt2tCJFJf6Y+oN7rEp6h+es54ZK/adcrkTp6gvLPXyyebDzF1/kG1HXMSZDAxQMxjSqTE9W6Q0yDGNTkes7Ge7Ctws2HSIBRsPUeDy0chu4YouTRjWJZMWKfGntQ3F0j4WTqxsQ6dL6hOZ1Cg8qU9kUqPwpD4/O53X1KHDW7Fu/ADrlhkYvE6CCc0rmkbqSIKpbessu6gDoQCWPV9j3TINS95ilJAff+PueDpehbfdlWhxSXonrHXSQGpAFE8R1s3TsG78EFNxHiFLEt72I/B0Gkcgo6ve8epcLJ7sjM6dWDdPI27LDIzuwwRtTfB2Gkf8uTejfPuv8Ce6Y3pcizbkPxQ4vTFxq42iQFKKnUfmhn/jf8zFHRqTZDXzyYaDeAMh1MYJXNk1k8EdG5MQVz9nKKxNsbafBYIhvssrZO76fL7PKySkwVktU7juvGyWbD3MrJX7I/6Msb2zePLKLpQ4XTGxj0USa9tQdUl9IpMahSf1iUxqFJ7Up4KiQHpKHMrC+6v2mrr1+WiahrL7OzSDCW/2YDydr8Hf4tyIw06Ihk8pL8S6dTbWzdMwFWxGM9nwqCMp73o9wfSOeserNdJAagCMBZuJX/c21q2zUQIefE3PwdN5PN6cIVF9i1okMX2yC/qx7P4S66bJWHZ/CQYDSqjqVztod/1IWXxrPB5/LYasH+LjzRwo8zPo+W+qvI7NYmRI50yu6NyYjk0SazFd/RfL+9nhUi8LNh5i/oZ89hd7qrXuovsvoKnNHBP7WCSxvA1VhdQnMqlReFKfyKRG4Ul9KsTHm7G78lBe7VPldbSkZvi63Uhpu9Ey9XusqrzFzbrhPaxb56IEvfianYOnyw14cwZH3TjE4RpI0jbVUyiAZecnJM8dQ9rUgVgds/C0H0HhVYsoHjkLrzo6pptHMc9oxpdzKSVD3yV0xzKURu2rtbqy7GVs5ti4vUYxGXlj6c5qrXNp50z+NrxzzDePYl3jxDhu6tOST+7tx3ltqzduwaRv8zCYjbWUTAghhBA1Ld4URPnh5Wqto+RcjOn830nzKJYpCoHG3Sm7+N8U3LCCsnP/grEsn6RFd5L2fh9syyeilBfonbJOSANJJ4qvlNQpF5P8ya0Yi/dQ1vfhio3xomcJNuqkdzxRjygKGDM7Qml+9VbcsgCj1U4Uj/0MVNTHbjWzeNOhaq33leMwdqs56usjIlMUSLJZ2HigpFrrLdqUL9uQEEII0UAoChWzrW1ZWL0VHQtj4jW1qBrNmkp5jzsovOZbioe8SyC9E/af/k3il3/UO1qdiL3BPuoJzWjFlz0IV2YvfK0HgkH+FOLkjs/+5a3em9tjyyuKQkO8VbWqjtWn1BOo1nrHlo/2+ojIZBsSQgghop+8phY1SjHga30JvtaXYCjeFTNjYknXQi9GM65z/6J3CtEAHD9RxSVBeVHVV6ycISDaT3SaphEKacSZDLh9Vb9lL9FqOr6+iG3HtoFEqwmnu+rjGSXEGX+xvhBCCCHqL03TwFMCBguEyqu+Yoy8phanL5TcWu8IdSY22mRCNGCaBkGPCzoMqd6K5niCO76J6hmiAiGN+evzGfCfr6vVPAIY1CkTl8cf1fURVaNp4PL4GdipSbXWK/eFePbTLZSUV+/KJSGEEELULaW8kPgfnkWb2BkC1WgeAXQYStAjs64KAdJAEqJBKA8Y0freU611NJ8b4/tXkPzRNZjyV9ZSMn0EQhoLNuYz5u3lPP7ZVsxGhYcu71Ctn3Frv2xC/tgYZFxEpgWC3Novp1rr9G6dyitLdjBs0o+88f3uat8CJ4QQQojapbiPYl/6OOnvnYNtxYsEW16ANvqdav0Mre89uP0yaYYQILewCdEgeDx+7Gk50OMaWP1B5BV6XAsDH8e19A3iV71G6qzh+Fr2x3XW/QQye9V+4FoSCGks2nKYN3/Yw56ictTGCTw3vDMXtk0jOTWB7YfLmLFiX8SfM7Z3Fq3S7ZQ4XXWQWjQEHo+f1ul2xvRuUeVt6Mkru/CT4xCTlu3m9WW7mbxqHxN6tmBcz+bHb5EUQgghRN1TPEXYVr9G/Lq3IejB234E7p53E0pvR3pKXPVeU6dl43V6az+0EA2A0hDv5fT7g5rT6dY7hqgDKSk25G9dwWw2kpxoRllwP6x+/9QL9rgWbehEikv9+P1B8LmI3/AuttWvYfAU4mt5Ia6z/0SgSW6dZT9TwZDGZyc0jtpl2LmtbysubJt+fEBEs9mIPdHKI3M2MH3F3lP+rLG9s3hqRBdcpZ6K+gjZzyqdyTbkOFzGpGW7WbK9gIQ4IxN6tmBC7+bYLbHRSJJtKDypT2RSo/CkPpFJjcKLlfoo3hLi175B/NpJKL4yvO2G4T7rDwRT2xxf5rRfUwsRIzIyElcCvU/2nDSQRL0WKye7qjKbjSQnmKBwJ8qyl2HLgoqZIeKSoMPQitvc0rIpLgv89kTncxG/4T1sq1/F4CnE02Yo7j5/JphSvdt26pKmaSzdWcjL3+axs8BNuww7t1Y2jgwnmUvVbDZiS7Cyq8DFpG/zWLQpn1JPgESriUGdMrm1Xzat0u24y6R5dCLZz352ptvQiY2k1HgzN/VpychuTbGYovuOcdmGwpP6RCY1Ck/qE5nUKLyor4/PRfz6tyte53qL8eZchuvsPxJMP/kQB2f0mlqIKCcNJNFgRf3J7jQoCsTFmbGZgxit9uOPBz0u3H4jXm/4gaEVXynxq/+Hbc3rEPTi6TQe91n3E7JXbwDh2rbuQAkvf7OT1ftLaJkaz53ntebi9o1O2jg60bH6GMxG7Fbz8cddHj8hfzBifWKR7Ge/VBPb0Mb8Ul7+Zicr9hbTLCmOO85vzaUdGkfcfhsq2YbCk/pEJjUKT+oTmdQovKitT9CPdfMU7D/9B0P5UbytB+A++/8IZHSJuOqZvqYWIlpJA0k0WFF7sqshigLJyTaKi93VPsEp7iPYV7yAdeOHYDBS3u0W3D3vRItLrp2wVbSrwM0rS/NYsr2ANJuZ285txfAumZiM1b+C40zqE0tkPzu1M9mGNE3jh91FvPxNHluPuGiXYed3/bI5t3Xq8Vsvo4VsQ+FJfSKTGoUn9YlMahRe1NVH07DkfYZ92T8xOXfga3YOrr4Pn/ZYn/KaUYifhWsgxcbgDEJEqWMnuNM50Wm2DMoueBJ391ux//gstlUvY934Ae7e91Le9XowWmo2bARHy7y8vmw389fnE2cycvu5rZjQqwU2y+nPenEm9RECzmwbUhSFvq3TOKdVKou3HOHV73Zx3+wN9GyRzH39c+jYJLFmwwohhBAxwJS/ioTvn8R88CcCqW0pvvwtfK0HVnSBTpO8ZhSiamqkgaSq6mDgBcAITHI4HE//6vmJwEWV39qAxg6HI6XyuSCwvvK5PQ6HY1hNZBJCVE0ouRWlg16mvMcd2Jf9k4Tv/o514/u4zvsrvlaXnNHJuCq8gRCTV+7j7R/34A9qjOnRnJvOySLVVrcNLCFqi0FRuLRjYy5u34g56/KZtGw313+wmqGdm3BXv2wa2WVbF0IIISIxFO/C/sMzWLd/RCg+g9ILn8bTaRwY5JoIIerKGe9tqqoagVeAgcA+YLmqqvMdDsemY8s4HI77T1j+HqDHCT+i3OFw5J5pDiHEmQlkdKH4ig+w7P4S+3ePk7zwBnxZF1J23l8Jpqs1/vs0TePLbUd58eudHCjx0r9tOvdemEOLlPga/11C1Admo4GxPZpxeafGvPnDHqau2s8XW49ywzlZTOjVgrgoH2hbCCGEOB2KrwzbyheJX/MGGEy4et9HeY870CwJekcTIubURLv2bGC7w+HYCaCq6lRgOLDpFMuPBx6rgd8rhKhpioKv9SX4si4gfsO72JZPJHXaIDydr8F19h/R4tNq5Nc4Dpfxn692sGpfMW0b2fnvmPac1TK1Rn62EPVdQpyJey/MYWS3przw9U7+u3QXc9cd5PcX5nBxu0ZRNz6SEEIIcVo0jbits7F//w+M7kN4OozB1ecBQvZMvZMJEbNqooHUHNh7wvf7gHNOtqCqqq2AbODLEx62qqq6AggATzscjrk1kEkIcSaMZsq734Kn/Ujsy/+NdcMHxG2bi/us+ynvesNpXypc6Pbx2ne7mLsunySriQcHtGV416aYDPKGWcSerNR4nruyMz/tLmLikp08+NFmerRI5v8uakP7xvKpqhBCiNhlOryOhG8fxZy/En/j7pRc9gaBzJ56xxIi5p3xLGyqqo4GBjscjlsqv78WOMfhcNx9kmUfAFo4HI57TnisucPh2K+qag4VjaVLHA7HjnC/MxQKacGgjHAWC4xGA8FgSO8Y9Vqd1OjIZoyLH8GQtwStcWeCg59Fy+pT5dWDIY2py/fy78+3Uu4Lcs05Lbn7orYkx5sjr3yGZBuKTGoUXl3UJxAMMWPVPiZ+vo3icj/X9mnFvRe3I9HaMMZ1kG0oPKlPZFKj8KQ+kUmNwmsw9XEdxbjkSZQ174O9EcGL/orWbTwotX+bd4OpkRC1zGw21uosbPuBrBO+b1H52MmMA3534gMOh2N/5b87VVVdQsX4SGEbSMGgFl3TUIpTiropR2tBndTI3Aouex/Lzk9IWPoYpvcup7zDVbjOfRgtPj3sqpvyS3n6821sPlTGWS1T+PPFbWmdbkPz+nF6/bWbG9mGqkJqFF5d1eeydo04t0Uyr363i/eW7WbBuoPcf2EOgzpk1Pvb2mQbCk/qE5nUKDypT2RSo/DqfX1CQawbP8D+wzMoATfl3W/FfdZ9aHFJUOypkwj1vkZC1JGMjFPPFFwTDaTlQDtVVbOpaByNAyb8eiFVVTsAqcCyEx5LBdwOh8Orqmoj4DzgXzWQSQhR0xQFX5vLKcy6EPuK54lf+wZxeZ/i6vMQnk7jwWD8xeIlHj+vLt3FrLUHSbNbeGpIBwaq9f+NsBB6So438+CAdlzRJZNnPt/GXz7ewtz1B/nzJe3ITrfpHU8IIYSocaYjG0hY8iDmw2vwtTifsn5PEExrp3csIcRJnPG1gA6HIwDcDXwGbAamOxyOjaqqPq6q6rATFh0HTHU4HCfee9YRWKGq6lrgKyrGQDrV4NtCiPrAYsd17iMUXbWIQHpHEr9+kJRZwzAdXgtUzK728aZDjHl7BbPXHWRsj2bMvLE3gzo0luaREFXUOTORtyf04IFL2uI47GLCeyt56Zs8yv1BvaMJIYQQNcPnwr7076TMuBxj6T5KBrxI8bAp0jwSoh474zGQ9OD3BzW5vDA2yKWkkelaI00jbuscEr57AqX8KIfU67n/6BUs2++lS9NEHrykHWoTfQcDlm0oMqlReHrXp9Dt46Vv8liw8RBNk+J4aGA7+raumRkRa4reNarvpD6RSY3Ck/pEJjUKr77Vx7LzUxK+fRRj2UHKO1+Dq8+DaNYUXTPVtxoJoZeMjMRaHQNJCBGrFAWvOhJ31sUcXPAoPRzv8Ky2kLW9H6X3Bf0wyBVHQpyxNJuFxwarDOuSyVOLtvL7WRu4vFNj7u/fhpQ6GIheCCGEqCmG0gMkfPMX4nYtIpDegaJLXyOQ2UvvWEKIKqr94eyFEFFt86FSrp25gxF7x/DPxv8hIzmRIRt+T/IX96F4ivSOJ0TU6NEimQ+v68VNfVry2ZYjjH17BZ9tPkxDvJJYCCFEjNFCWDe8R+qUi7Ds+5ayvo9QNOYTaR4J0cDIFUhCiNPi8Qd5/fvdfLhyH2k2C/8a1omL2l1ASWAYtpUvYVv1CpY9Syg7/+942w0HuRpJiDMWZzJw53mtGdg+gycXbeUvH2/h0y2HeeCStmQmWfWOJ4QQQvyGwZlH4ld/wnLgB3wt+lF60TOEklrqHUsIcRrkCiQhRLWt2ONk/HsreX/FPq7oksn0G3pzUbtGFU+arLjP+RNFYz8hmJhF0uK7SVp4A4bS/fqGFiKKtM2w8+b4XO7vn8OKPU6uemcl01fvJyRXIwkhhKgvQkHi17xB2rSBmI5upPSiZykeNlmaR0I0YNJAEkJUWZk3wFOLtnLnjHUAvDqmG38Z1J5E628vZgymd8Q5ah5l5/8Ny/7vSZ1yMdaNH4K8wRWiRhgNChN6tWDaDb3p1iyJZ7/cwW1T17K3qFzvaEIIIWKcsXArKbNHkPDd3/G1OJ+i8V/i6TRerkgXooGTBpIQokp+2l3E+HdXMn9DPtf2bsGU63rRu2VK+JUMRsq730Lh+C8INM4lcckDJC+4FkPZgTrJLEQsaJZs5cVRXXhscHt2FLiY8J5cjSSEEEInQT+2FS+ROm0wxuI8Sga+RMnlbxNKaKp3MiFEDZAGkhAirHJ/kGe/2M7vZq7HYjLw5vhcfn9hDlazsco/I5TUkuLhUyi94EnMB34kdcoA4rbMlKuRhKghiqIwtHMmU6/vTW6LZJ79cge/m7megyUevaMJIYSIEcbCraTMGo79x2fw5lxK4fiv8LYfIVcdCRFFpIEkhDiltfuLufq9lUxfc4BxPZvz4bU96dI06fR+mGLA0/UGCq9aRDBdJemL+0j65BYU95GaDS1EDGuSGMeLI7vw8MB2bDpYyvh3VzJ33UGZqU0IIUTt0ULEr3mD1OmXYSzdR/Hg/1F66atotkZ6JxNC1DBpIAkhfsMbCPHSNzu5bdpaAiGN18Z2448XtanWVUenEkrJxnnlTMrOfRTLniWkTbkYy/YFNZBaCAEVVyON6NaUKdf3omOTBJ5avI375mzgSJlX72hCCCGijKFkH8nzrqoY6yjrAgrHf4GvzRC9Ywkhaok0kIQQv7DlUCnXfbCK95bvY1iXTKZc34teWSk1+0sMRsp73E7R2E8JJrUk+bM7SPzsLhRPUc3+HiFiWLNkK6+M6cb/XdSGlXuLueqdlXy6+bDesYQQQkQDTSNu83RSpw7AdHgdpRc9R8nlb6HZMvROJoSoRb+dOkkIEZOCIY33lu/lf9/vJjXezPMju3Bedlrt/s60djhHzcO26lVsy/+N+eBPlA54AX+L82r19woRKwyKwlU9m9M3O42/f+rg0Y+3sHRnAQ8OaEdCnLwEEEIIUX2K+yiJSx4gLu8zfM3OofSSiYSSWuodSwhRB+QKJCEE+SUe7pqxjv8u3cVFbdOZen2vWm8eHWcw4e59D85R89HMdpLnjcP+/ZMQlNtthKgpLVPj+d9V3bn93FZ87jjChPdWsnpfsd6xhBBCNDCWvMWkTb0Ey+6vKDv3UYqvnCHNIyFiiDSQhIhxFW8mV7HlUBmPDW7PP4Z2JDneXOc5Ao27UTT2Ezydr8G2+jVSZg7DWLitznMIEa1MBoVb+rZi0vhcjAaFO6av5dWleQSCIb2jCSGEqO8C5SR8/QjJH99I0J5J0diPKe9xOyjydlKIWCJ7vBAxyuUL8PinDh5asJmWqfF8cG1PhnbORNFzqlWzjbL+/6T48rcwlh0kdfpgrOvfBZlBSoga06VpUuX+3oS3ftzLzVPXsqeoXO9YQggh6iljwRZSZwwlfsO7uHNvxzl6PsH0DnrHEkLoQBpIQsSgjQdLuOb9VSzcdIib+rRk0rjuZKXG6x3rOF/2IArHfY6/eR8Sv3mEpIU3oLiP6B1LiKhht5h49FKVZ67oyD5nOVe/t5K56w6iSbNWCCHEMZqGdf07pM4YgqG8EOcVH+A671EwxumdTAihE2kgCRFDgiGNt37Yw81T1hAIarw2tjt3ntcak7H+HQo0e2OKh75P2fl/x7JvKWlTB2Le+43esYSIKhe3z2Dydb3o0iyJpxZv48GPNlPqCegdSwghhM6U8kKSPr6ZxG/+gq/5uRSOW4S/ZX+9YwkhdFb/3jUKIWrF4VIvd85Yx6vf7eKSyjeNPVok6x0rPMVAefebKRqzkFB8Osnzr8a+7GkIyRtcIWpKk8Q4Xhndld9fkM3XOwq45v2VbDhYoncsIYQQOjHv+47UaQOx7FlC2fl/o2Tou2i2DL1jCSHqAWkgCREDvs8r5Or3V7HlUCl/G6zy5JAOJFobzhTewfQOFI1egKfTOGyrXiZlzmgMpfv1jiVE1DAoCteelcWkcd0BuGXqWt5fvpeQ3NImhBCxI+jHvuwfJM8bh2ZJpGj0R5R3v0UGyhZCHCdHAyEasGPjXZ9q3OtAMMRL3+Rx7+wNNLJbeO/qngzp3ETfgbJPlzmesouepWTgyxgLNpM6bRCWvEVhV4lUHyHEL1UMsN2LC9uk8+I3edw/ZwNFbl/YdWQ/E0KI+q0qx2lD6QFS5o7Gtuq/eDqNp2jMxwQzOtdNQCFEg6E0xAEz/f6g5nS69Y4h6kBKig35W/+SooDVakYxGbFbzccfd3n8aIEgHo8fTYP8Eg+PLNzCugMljOzWlPv752A1G3VMXnMMzjySFt2F+ch63N1uxnXuw8cHdDxWn3hTEKPVfnydoMdFecB4vD7iZ7KfhReL9dE0jZlrD/L8kh0kx5t54vIO9MpKOf58VY9DokIsbkPVJTUKT+oTmdTol6rzesiy6wsSP78XQgHKLvoX3nbDdEqtL9mGhKiQkZG4Euh9sucazj0sQgjMZiO2BCu7Cly88a2DxZsOUeoJkGg1MbBTE27tl0PrdDufrN7HXxduJhDSeGpIBwZ1aKx39BoVSsnGOWou9u+fwrbuTcwHf6Jk0H8xZrQlOcEEhTtRlr0EWxaCtwTikjB2GIK97z3Y03IoLgvg9wf1/t8Qot5SFIUxuc3o1iyJhxds5q4Z67ilTytu6tMSa5ypSschd5lH9jMhhNCB2Wys2uuhEg+Wpf/Etuq/BNI7UTL4NYIpOXrHF0LUY3IFkqjX5JOAn5nNRuyJVh6es54ZK/adcrn2TRLYeqiMDk0SeGpIR1qmxtdhyrpn2fkpiV/+EUULwtCJKHlfw+oPTr1Cj2vRhk6kuNQvb24ryX4WXqzXx+0L8vTn2/hk82F6t0zhhfE9eP6LbWGPQ2N7Z/HUiC64SqWJBLINVYXUKDypT2RSowpms5HkRDPKgvvCvx7qPAqtZD/K3h8o73Q1Zf3+Bqbofs0YiWxDQlSo9SuQVFUdDLwAGIFJDofj6V89fwPwLHBs1NuXHQ7HpMrnrgf+Uvn4kw6H492ayCRENFEUsCVEbh4BbD1URtuMBGbccS5elyfqbyPx5QzGmdGF1C/vQZl9a+QVVr+PAiQP+Q8FzmDU10eIM2WzGPn7ZSpnt0rhX1/s4OJ/f403EAq7zvQVewF48soulDhdsp8JIUQdUBRITjBFbh4BbJyFYjChDX8VV9YVIMdpIUQVnPEg2qqqGoFXgMuATsB4VVU7nWTRaQ6HI7fyv2PNozTgMeAc4GzgMVVVU880kxDRxmo1s+uoK2Lz6JjtR8o4WOIhLs4ceeEoYGmcDZdPrPoKq9+Hwp0xUx8hzpSiKIzpncXzV+VGbB4dM33FXnYVuGQ/E0KIOmK1mqFgR+Tm0TGhADTvKcdpIUSV1cQsbGcD2x0Ox06Hw+EDpgLDq7jupcBih8NR6HA4ioDFwOAayCREVFFMRt5YurNa60z6Ng9DlAyaHUm8KYiy/LVqraMsexmbWW6tEaKqFJORz7ccqtY6sXQcEkIIvcWbgig/vFytdeT1kBCiOmqigdQc2HvC9/sqH/u1UaqqrlNVdaaqqlnVXFeImKUoYLeaWbypem/cFm3Kx241R/3U2opCxewiWxZWb8UtCzBa7VFfHyFqghyHhBCifpPXQ0LoqyGOLX066moWto+AKQ6Hw6uq6u3Au8DFp/vDjEaFlBRbjYUT9ZfRaJC/daVST+C0lk9OjpH6eUtOa/mYqU8Ysp+FJ/X5mRyHTo9sQ5FJjcKT+kQmNaokr4dOm2xD4nTkl3i4f/paMhLieHFcrt5xal1NNJD2A1knfN+CnwfLBsDhcBSc8O0k4F8nrNv/V+suifQLg0FNRsiPETIbQsUnSo0aJZJoNeF0+6u8XqK1YvcuLnZH9QC2x+pDXBKUF1V9xbgkIPrrUxWyn4Un9ZHj0JmSbSgyqVF4Up/IYr1G8nrozMX6NiSqb+VeJw8v2Ey5P8hTQzpGzfaTkZF4yudq4ha25UA7VVWzVVW1AOOA+ScuoKpq0xO+HQZsrvz6M2CQqqqplYNnD6p8TAhRSdPA5fFzcYfG1VpvUKdMXB5/1L8Y0DQIelzQYUj1VuwwlKBHZocSoiqOHYcGdmpSrfVi5TgkhBB6k9dDQtQdTdOYvHIfv5uxjoQ4E+9c3YN+bdL1jlUnzriB5HA4AsDdVDR+NgPTHQ7HRlVVH1dVdVjlYr9XVXWjqqprgd8DN1SuWwg8QUUTajnweOVjQogT7DlSxtq9zmqtc2u/bEL+2BgUsTxgROt7T7XW0c66DbdfBvcVoqq0QJBb++VUa51YOg4JIYTeygNGtKa51VpH63uPvB4SohrK/UH+snALE5fspF+bdN69ugc56Xa9Y9WZGhkDyeFwfAx8/KvH/nrC1w8BD51i3beAt2oihxDRaNmuQh5ZsAWDQeH8tuks3V4QcZ2xvbNolW6nxOmqg4T683j82NNyoMc1VZ+6dsG9+Af9D+xNIy8rhMDj8dM63c6Y3i2YsWJfxOUVYPGmQ4zp0rjyOyGEELUmFMSw5FmUFS9AfBqUV+Ez+R7XQlo2Xqe39vMJEQX2FpXzp/kb2XnUzV3nt+b6s7MwxNgI9DVxC5sQohZomsZ7P+3lvtkbyEyK48Pre/H2jWcztndW2PXG9s7iqRFdcJd5YuZyZE2D4rIA2tDnK14MhdPjWrTR78CRLaRMuxzTgZ/qIqIQDZ6mgbvMwz9GdI14HBrRozmXdsnkX585eGTBFsrlKiQhhKg1iqeI5IXXYVvxAt5O49DuXVu110NDJ1a8foqR14tCnIlvdxRw3YerOFrm48VRXbjxnJYx1zwCUBridHN+f1CLlgGqRHixOphduT/IE59tZbHjCAPaZ/DXwe2JNxsxm43YEqzsKnAx6ds8Fm3Kp9QTINFqYlCnTG7tl02rdDvuMg/+GHzDZjYbSU4wQeFOlGUvw5YFFbOLxCVBh6EVt7mlZVNcFiCUv4mkT27GWLqPsn6P4+l8LbE6h22s7mdVJfX5paoeh1yl5Uz6bhf/XbqLthl2nh3eiebJ8XrH14VsQ5FJjcKT+kQWqzUyHt1E8sc3Y3AdouyCx/F0uhqzxVTl10Ox+HrxVGJ1GxLhhTSNN5ft4fVlu1EbJ/CvYZ1olmzVO1atyshIXAn0Ptlz0kAS9VosHsgPFHv4v3kb2X7Exe/6ZXPdWS1QTmhsKArExZkxmI3Yrebjj7s8fkL+IF5vbA9Ye6w+NnMQo/Xn+5GDHhduv/EX9VG8xSQuvoe43V9S3nEcZRc+BcY4nZLrJxb3s+qQ+vxWdY5D3+cV8peFWzAo8NTQjpzTKlWn1PqRbSgyqVF4Up/IYrFGlu0LSPrifkJxyZQMfp1AZs/jz1Xn9ZCoEIvbkAjP7Qvy2CdbWLK9gCGdGvPggHZYzdE/Zpg0kESDFWsH8uV7injoo82ENHhySAfOzU4Lu7yiQHKyTaZePYUq1ScUxPbTv7GvfBF/kx6UDH6dUEJsjYsUa/tZdUl9wqvKfra3qJz/m7eRXYVu7u6XzTW9f9kYj3ayDUUmNQpP6hNZTNVIC1W8dlnxAv7MXhWvXeynniVTXi9WTUxtQyKifc6K1y55BW7uvTCH8T2bx8xrl3ANJBkDSYh64NhUkPfMXE+a3cI7V/eI2DyqWO+X/4pfqlJ9DEbcff5M8eD/YSpwkDJjCKaDy+sknxDRoCr7WVZqPG9P6MFF7Rrx4jd5PPrxFjxy24QQQlSb4isl6eNbsK94gfKOV+G8cnrY5hHI60Uhqmv5niJu+HA1R8p8vDiyKxN6xdYHX+FIA0kInXn8Qf72qeP4VJBvT8ilZWpsjhOiJ1+bIRSN/gjNbCNl7lisG6o4m5sQokpsFiP/HNqRu85vzaItR7hpyhr2F5frHUsIIRoMgzOPlJnDsez+gtJ+j1N20XMxeeu9ELVF0zSmrdr/84f6E3pwTuvYu/U+HGkgCaGj/BIPt01by8ebDnP7ua14Zlgn7BaT3rFiVjBdxTl6Ab4W55P49YMkLHkQgn69YwkRNRRF4cZzWjJxZBfyS7xc/8FqVu516h1LCCHqPfOer0mdORSD+zDFwybj6XZTzE7+IURt8AVCPLloK899tYPzcyo+1M+SD/V/QxpIQuhk7f5irv9wNXuKynlueGdu6dsqJqeCrG80awolQ97B3fMu4jd+QPJHE1A8RXrHEiKqnJedxrtX9yDVZuZ3M9cze91BvSMJIUT9pGnEr3mD5AXXErJnUjRmIf4W5+mdSoiocrTMyx3T1zJ/wyFu7tOSfw2XD/VPRRpIQuhgwcZ87pyxDrvFyNsTenBh23S9I4kTGYy4+j5MyYDnMR9cSeqMoRgLt+qdSoiocmxcpLNbpvDPxdt47svtBEIyQIcQQhwX8JD4xf0kfPd3fNmDKBo1n1ByK71TCRFVNuaXcv2Hq9l2xMXTV3TkjvNay4f6YUgDSYg6FAxpvPTNTv7+6Va6N0/m7Qk9yE636R1LnIJXHY1zxAwUv5uUmcOw7PpC70hCRJWEOBMTR3RhQq/mTFt9gPtmr6fEI7eNCiGEwZVPypzRWB0zcZ39R0oGvw4Wu96xhIgqH286xG1T12AyKLw5PpdL2mfoHanekwaSEHXE5Qvwp3kbeW/5PkZ1b8pLI7uQHG/WO5aIIJDZi6IxCwkmtyZp4Q3Er35NpjERogYZDQr392/Do4Pas3JvMTdOXsPuQplGWQgRu0yH1pAyfQimwq0UX/YG7rPuB0XetglRU459qP/YJw66Nkvi3at70r5xgt6xGgQ5EglRBw4Ue7hlylq+zyvkTxe35cEB7TAZZfdrKEKJzXCOnI2vzWUkfP8kiV/+AYJevWMJEVWGdc3k1THdKPEEuHHyGn7cJWOPCSFij2X7AlLmjAKjhaLR8/DlXKZ3JCGiitsX5M/zNx3/UP/lUV1JscmH+lUl72CFqGVr9xdzw4eryS/18MLIrozt0UzvSOJ0mG2UXPoarrPux7plBilzr0JxH9E7lRBRJbdFMu9e3YMmiXHcO3s901btR5Mr/oQQsUDTsC1/nuTP7iCQ0ZWi0R8RTO+odyohokp+iYdbpq5h6c4C/nRxG/lQ/zRItYSoRccGy060mnh7Qg/OaZ2qdyRxJhQD7rP/SMmgVzEd3VAxuPbRTXqnEiKqNEu2Mml8d87LSee5r3bwz8+34Q+G9I4lhBC1J+AhcfE92H96Dk/7kTiHT0WzNdI7lRBRZcPBEq7/cDUHij1MHNGFsT2a6x2pQZIGkhC1IBjSePHrisGyc5sn89b4XFqnyWDZ0cLb7gqcI2aDFiR11nAsOz/RO5IQUcVuMfHs8E7ccHYWc9blc/fM9TjdMri2ECL6KO4jpMwdi3XbXFznPEDpgBfAZNU7lhBRZdGWw9wxfR1Ws5G3JuRybnaa3pEaLGkgCVHDXL4A/zdvI++v2Mfo7k15UQbLjkqBxt1wjllIIE0l+ZNbiV/5sgyuLUQNMigKv+uXzeOXq2w4WMINk1ezq0AG1xZCRA/j0U2kzhiKqWATxYP/h7v3PSDThwtRYzRN4/Xvd/HIwi10bJLAOxNyyUmX2QzPhDSQhKhBB4o93DxlDcvyCvnzJW15QO6rjWohexOcI2bgaTechB+eJuGr/4OgT+9YQkSVyzo24bWx3Sn3B7lpyhqW75HBtYUQDZ9l1+ekzB4BWgDniNn42gzRO5IQUcXjD/KXhVt4Y9kehnRuwiuju5Fqs+gdq8GTd7ZC1JD1B0q44cPVHC718cKorozJlcGyY4IpntKBL+HqfS/xm6eR/NE1KB6n3qmEiCpdmyXx9oQeNEqwcM+sDcxfn693JCGEOD2aRvya10laeCPBlDY4Ry8g0Lib3qmEiCpHXT7unLGOxY4j3N0vm8cubY/FJK2PmiBVFKIGfLH1CHfOWIfNUnFf7TmtZLDsmKIYcJ/zJ0oGPI/54HJSZg3HULxL71RCRJVmyVbeGp/LWVkpPLFoKy99k0dIbhsVQjQkQR8JS/5MwneP42tzGc4RswglNNU7lRBRZevhMm74cDXbj7h4Zlgnrj87C0VuDa0x0kAS4gxomsY7P+7hwY82ozZO4O0JMlh2LPOqoykePgVDeQGpM6/AdOAnvSMJEVUS4kxMHNmFUd2b8t7yvTz00WY8/qDesYQQIiLFU0TyR1cTv2kKrl73UHLpa2CO1zuWEFHl6+0F3DJ1DZqmMWlcLhe1k9kMa5o0kIQ4TYFgiKcWbeOVpbsYpGbw3zFyX60Af7M+OEfPJxSXQsq8ccRtnaN3JCGiismg8MAlbbm/fw5fbTvKHdPXcdQlY48JIeovgzOPlJnDMB9cScmA53H3eQAUeRsmRE3RNI3JK/fxp3kbaZ1m452re6A2SdA7VlSSI5cQp6HUE+D3szcwb0M+N/VpyRNDOhAn99WKSsGUHJyj5+PP7EnS4nuw/fQfmaFNiBqkKAoTerXg2eGd2HHUxY0frmb7UZfesYQQ4jdMB1eQOmsYBq8T5/CpeNXRekcSIqoEQhrPfrmDiUt20r9dI16/qjsZCXF6x4pappr4IaqqDgZeAIzAJIfD8fSvnv8DcAsQAI4ANzkcjt2VzwWB9ZWL7nE4HMNqIpMQtWV/cTn3z97IXmc5jw1uz9DOmXpHEvWQZk2leNhkEpc8gH35fzAW51F68XNglBOaEDXlwraNeGNcd/4wdyO3TFnDP6/oSN/WaXrHEkIIAOK2fUTiF/cRTGhKydD3CKbk6B1JiKji9gV5ZOFmlu4s5OpeLfj9hdkYZLyjWnXGl0yoqmoEXgEuAzoB41VV7fSrxVYDvR0ORzdgJvCvE54rdzgcuZX/SfNI1GsbDpZw0+Q1HHX5eHl0V2keifCMFkov/g+ucx7AunUOKfPGo5QX6p1KiKjSoUkib0/oQbNkK/fP3sDMNQf0jiSEiHWaRvyqV0hadCeBxt1wjpovzSMhatiRMi+3TVvL93mF/PmSttzXP0eaR3WgJu65ORvY7nA4djocDh8wFRh+4gIOh+Mrh8Phrvz2B6BFDfxeIerUF1uPcMf0dcSbK2Za65WVonck0RAoCu7e91Ay6FVMh9eSOvMKjEU79E4lRFRpkhjHpHG59M1O45kvtvOfr3YQDMlto0IIHQT9JCx5gIRl/8TTbjjOYVPQ4uXKSCFq0vYjLm6cvIY9RW7+fWVnxuQ20ztSzKiJBlJzYO8J3++rfOxUbgY+OeF7q6qqK1RV/UFV1StrII8QNUrTNN79aa/MtCbOiLfdFTivnI7id5Eyaxjm/d/rHUmIqGKzGHlueGfG92zOlFX7+dO8jbh9MkObEKLuKL5Skj++gfhNk3H1uofSgS+Byap3LCGiyg+7Crll6hpCmsYbV+Vyfk663pFiSo2MgVRVqqpeA/QGLjzh4VYOh2O/qqo5wJeqqq53OBxhP543GhVSUuQNfCwwGg26/q39wRCPfbSJGSv3MaRrJs+M6Eqc2ahbnpPRu0b1Xb2qT0o/gpmfY5p2FcnzryZ4xctoXcbonap+1agekvpEVp9q9PiIrqjNknl84SZ+N2s9r1/Ti4xEfcceq0/1qa+kRuFJfSLTvUYl+zDNGwdHHASGvIAl91rq09y8utenAZAa1X/TV+zlrx9tol1GAq9f24umydKgrWs10UDaD2Sd8H2Lysd+QVXVAcAjwIUOh8N77HGHw7G/8t+dqqouAXoAYRtIwaCG0+kOt4iIEikpNt3+1qWeAA98tInle5zc1Kclt5/binKXl3Jd0pyanjVqCOpffRqhXDmbpE9uxTLvdlz5O3H3uht0vGe7/tWofpH6RFbfajREbUSSqTMPL9jMqNe+54WRXclO1+9NQX2rT30kNQpP6hOZnjUyHdlA0sLr0fxuSq54H3/WBVDP/l6yDUUmNaq/QprGq0t38c5Pe+nTOpV/Du1IvBaSv1ctychIPOVzNXEL23Kgnaqq2aqqWoBxwPwTF1BVtQfwP2CYw+E4fMLjqaqqxlV+3Qg4D9hUA5mEOCP5JR5umbqG1fuK+eul7bnzvNYyKJuoMVpcMsVXfICn/UjsPz5DwpIHIBTQO5YQUaVfm3T+d1V3vIEQN09Zw8q9Tr0jCSGikGXXF6TMHgmKEefIORXNIyFEjfEGQvxl4Rbe+WkvI7plMvHKziTE1emNVOIEZ9xAcjgcAeBu4DNgMzDd4XBsVFX1cVVVj82q9iyQAMxQVXWNqqrHGkwdgRWqqq4FvgKedjgc0kASutp2pIybpqzhUKmXF0d14YouMtOaqAVGC6UDXsDV+17iN00meeENKL4yvVMJEVU6ZVbM0NbIbuGeWev5dPPhyCsJIUQVWde/S9LHNxJIbYNz9EcE0zvoHUmIqOJ0+/ndjHUsdhzh9xdk89CAdpiMNXENjDhdiqY1vFlK/P6gJperxYa6vpR0+Z4i/jRvE3aLkRdGdqVthr3Ofvfpksttw2sI9bFumkzCkocIpHegZOi7hOx127RsCDXSk9QnsvpeoxKPnz/N28SqfcX87vzWXH92FkodXlVa3+tTH0iNwpP6RFanNdJC2L97Etva1/G2HkjJwJfBUr9fM8o2FJnUqH7ZU1TOfbPXc6jUy98v68AANUPvSDEjIyNxJRVjV/+GtO+EqPTp5sP8ftYGmiTG8eb43AbRPBLRwdNpAsVD3sFYvIuUmcMwFmzRO5IQUSXJaualUV25tEMGryzdxT8/30Yg1PA+QBNC1AP+cpI+vR3b2tdxd72Rkssm1fvmkRANzdr9xdw0eTWl3iCvju0uzaN6RBpIIuZpmsa7P+3l0Y+30L15EpPG5ZKZJCP6i7rlb3URzhGzIRQkZfYIzPu+0zuSEFHFYjLw+OUduPGcLOasy+f/5m7E7QvqHUsI0YAo5YWkzLsKy85PKTv/b7gueAIM9Wt2XiEaui+3HuGuGetIjjfz9oRcujVL0juSOIE0kERMC4Y0nv1yBy9/m8cgNYMXR3Yl0SqDsgl9BDM64xw9n1BCM5I/uoY4x0y9IwkRVQyKwl3nZ/PQwHb8sKuQ26et5WiZN/KKQoiYZyjeTcqs4ZiObqTkstcp736L3pGEiDpTV+3nwY8206FJIm+Oz6VFSrzekcSvSANJxCyPP8iDH21ixpoDXNO7BU8M6YDFJLuE0FcosTnOkbPxNz2bpM/vw7b8eWiAY9UJUZ+N7NaUf1/Zhd1Fbm6cvIYdR116RxJC1GOmw+tInTUcg6cI5/Bp+HIu0zuSEFElpGm88PVO/v3VDi5sm84ro7uSEm/WO5Y4CXm3LGKSs9zPXTPW8/X2Av54URvuvTAHQx0OqCpEOFpcMsVXvI9HHY39p+dI+Or/IOjXO5YQUeW8nDRev6o7/pDGLVPXsGKPU+9IQoh6yLL7S1LmjEYzxeMcNZdA05OOKyuEOE2+QIhHF27hgxX7GJvbjKev6ITVLLeG1lfSQBIxZ39xOTdPWYPjcClPX9GRcT2b6x1JiN8yWii9ZCKu3vcRv3kayQtvQPGV6p1KiKjSoUkib0/IJSMhjntmrefjTYf0jiSEqEfiNk8jaeGNBFJyKBo1j2BqW70jCRFVSj0Bfj97PYscR7inXzb/d3EbjAb5UL8+kwaSiCmbD5Vy0+Q1OMv9vDK6Gxe3lxH9RT2mKLjP+T9KL3oO876lpMweiaHsoN6phIgqTZOsvDkul+7Nk3jsEwdv/bAHTW4bFSK2aRq25RNJ+vKP+FucT/GImWj2xnqnEiKqHCr1cuu0NazdX8Ljl6tcd3YWitwRUu9JA0nEjO/zKgZMjTMZmDQul9wWyXpHEqJKPJ3GUTz0XQwle0iZNQxjwRa9IwkRVRKtJl4c2ZXBHRvz6ne7+MfibQRC0kQSIiaFAiQseQD7T//G02EMxUPeQbMk6J1KiKiy/aiLmyavJr/Eywsju3BZxyZ6RxJVJA0kERPmb8jnD3M2kJUSz1vjc8lOt+kdSYhq8bfsj3PkHAiFSJk9EvP+ZXpHEiKqWEwGHr9M5cZzspi7Pp8/zdtIuT+odywhRF3yu0n6+GbiN03G1fteSi/+DxhlIF8hatLKvU5unboGDXhjXHfObpWqdyRRDdJAElFN0zTeWLabJz7bSu+WKfzvqu40SojTO5YQpyXYqBPOUfMI2ZuQPP9q4rZ9pHckIaKKoijcdX42Dw5oy/d5hdw5fR1Fbp/esYQQdUBxHyVl7hgse76i9MKncZ/zJ5DbaYSoUYu2HOaeWevJSIjjrfG5tMuQq/saGmkgiagVCGn8Y/E2Xv9+N0M6NWbiiC4kxJn0jiXEGQkltcA5cjaBJrkkLbqT+DVv6B1JiKgzqnsz/jWsE9uPurh5yhr2Ocv1jiSEqEUGZx6ps4ZjKnRQctkkPF2u0TuSEFHnwxX7eGThFrpkJjJpXHcyk6x6RxKnQRpIIiqV+4P8ad5G5q7P58ZzsnhssIrZKJu7iA6aNRXnsMl421xOwnd/x770cdBCescSIqpc2LYR/x3TjRJPgJsmr2FjvsyCKEQ0Mh1aTersK1F8pTiHT8OXPUjvSEJElZCm8Z+vdvD81zsZ0L4RL43uRpJVbg1tqOQdtYg6hW4ft09by/d5hTw4oC13nZ8tI/qL6GOyUjLoVdxdb8S29nUSF90NQa/eqYSIKt2aJTFpfC7xZgN3TFvLd3mFekcSQtQgy67PSZk7Fs2cgHPUXAKZvfSOJERU8QZCPLJgM1NW7Wd8z+Y8NbQjcSZpQTRk8tcTUWVvUTk3T1nDzgI3/xrWmVHdm+kdSYjaYzDi6vc4ZX0fwbp9PskfXYPiLdY7lRBRpXWajTcn9KBVmo0/ztnA/A35ekcSQtQA68YPSfr4JgJp7SkaNZdgSo7ekYSIKiUeP/fMWs/nW49y34U5/OGiNhjkQ/0GTxpIImpsOFjCTVPWUOYN8uqYblzYNl3vSELUPkWhvOedlAx8CfPBFaTMHomh7IDeqYSIKo3sFv53VTfOapnKE59tZdKy3WiapncsIcTp0DRsPz5L4pIH8LXsj3P4dDRbht6phIgq+SUebpmylg0HS3hqSAeu7t1C70iihkgDSUSFb3YUcMf0ddgtRt4cn0vXZkl6RxKiTnnbj6B46PsYSveTMms4xoItekcSIqrYLSYmjujMkE6N+d/3u/nH4m0EQtJEEqJBCfpJ+PL/sK94gfKO4yi5/C2w2PVOJURU2Xq4jJumrOGIy8tLo7oyqENjvSOJGiQNJNHgzV57gD/N20hOuo03x+fSMjVe70hC6MKfdT7OkbMhFCJlzijM+5fpHUmIqGIyGnhssMpN52Qxd30+f5q3kXJ/UO9YQoiq8LlI/vhG4rdMw3XWHyi76FkwyOy8QtSkn3YXcdu0tSjAG+Ny6ZWVonckUcOkgSQaLE3TeHVpHv/8fDt9W6fxv6u6k2636B1LCF0FG3XCOWoeIVtjkudfTdy2j/SOJERUURSFO8/P5oFL2vJ9XiF3zVhHkdundywhRBiK6zApc0dj3vstpRc9i/vsP4CMxSJEjfpk8yHunb2BzKQ43prQg7aN5Oq+aCQNJFFvHTuvn+z8HgiG+PtnW3nrx70M75rJc1d2Jt5srNuAQtRToaQWOEfOJtCkO4mL7iJ+7aRTLhtuPxNCnNro3GY8c0Unth1xcfOUNexzlp90OdnHhKh94fYzY9EOUmcNx1S0nZLL38LTaXzdhhMiymmaxrs/7eWvHzvo3jyJN67KpUlinN6xRC2RBpKoVxQF4uPN2BKtNGqUiNlspFGjRGyJVuLjzSgKuHwB7puzgYUbD3Hbua14ZGA7TAZ5ZS7EiTRrKs5hk/HlDCZh6d+wf/cEaCHg5/0sLdHwi/0sLdFwfD8TQkTWv10jXhndlRJPgJunrGFTfilQtXOZEOLMVOVcZspfScrsK1ECbpxXzsDX+hK9YwsRVYIhjWe/3MHL3+YxSM3gxZFdSbTKraHRTP66ot4wm43YEqzsKnDxxrcOFm86RKknQKLVxMBOTbi1Xw42i5Hbp61i+5EyHr20PcO6ZOodW4j6yxRPyaWvkbD0MWxr/ofBlY/n0hdITrFD4U6UZS/BloXgLYG4JIwdhmDvew/2tByKywL4ZWwXISLq3jyZSeNzuXfWeu6YvpZnr+zCoO7Nw57LWqfbcZd5ZB8T4jSZzUaSE0zhz2VHtsDc2wkmZFJ8xQeEklvrHVuIqOLxB3n04y0s2V7Atb1bcPcF2RjkE5KopzTEaWj9/qDmdLr1jiFqkNlsxJ5o5eE565mxYt8pl7NZKm5Te/bKzpwtg7IBkJJiQ/aHU5P6AJpG/OpXSVj2D7RW56EkNYf100+9fI9r0YZOpLjUL29wkW2oKqRGcNTl4/45G9h2xEXPliks31V0ymXH9s7iqRFdcJVKE+kY2YbCk/r8zGw2kpxoRllwH6z+IMySClrznpQMeQefObWu4tVbsg1FJjWqOme5nz/O3cj6AyX84aI2jOvZXO9IogZlZCSuBHqf7LkauQJJVdXBwAuAEZjkcDie/tXzccB7QC+gALjK4XDsqnzuIeBmIAj83uFwfFYTmUTDoShgS4jcPAJw+4IM6NiEgd2aU+J00QD7n0LUPUXB0+su7I2zUOb/7vitbKe0+n0UIHnIfyhwBmU/E6IKMhIsTL6tL5e98E3Y5hHA9BV7AXjyyi5yLhOiGhQFkhNMVWgeAWgojdqT1DiTAqdX9jMhasj+4nLunbWBgyUenr6iIxe3z9A7kqhDZzwGkqqqRuAV4DKgEzBeVdVOv1rsZqDI4XC0BSYCz1Su2wkYB3QGBgP/rfx5IoZYrWZ2HXVFbB4d8/nmQ+wqcBEXZ67lZEJED6vVDM16RG4eHbP6fSjcKfuZEFVktZo5WurlUIm3SstPX7FXzmVCVJPVaoaCHVVoHlVaO0XOZULUoC2HSrlp8hqKyv28PLqbNI9iUE0Mon02sN3hcOx0OBw+YCow/FfLDAferfx6JnCJqqpK5eNTHQ6H1+Fw5AHbK3+eiCGKycgbS3dWa51J3+ZhkFnXhKiyeFMQ5YeXq7WOsuxlbGa5vUaIqpBzmRC1T85lQuhn2a5Cbp+2DovRwKRxufRokax3JKGDmmggNQf2nvD9vsrHTrqMw+EIAMVAehXXFVFMUcBuNbN406FqrbdoUz52q8xkI0RVKAoYrfaKQUarY8sCjFa77GdCRCDnMiFqn5zLhNDPgo353D9nI81TrLw1IZfsdJvekYROGuQsbEajQkqKbLTRpNQTOK3lk5NlOzAaDbI/hCH1OYG35LSWj/X9TLahyKRGFeRcdvpkGwpP6nMCOZedFtmGIpMa/Zamabz69U4mfrGNc9uk8/K4HiRaG2QLQdSQmvjr7weyTvi+ReVjJ1tmn6qqJiCZisG0q7LubwSDmoyQHyUUBRo1SiTRasLp9ld5vWMHruJid8wPiigzRoQn9fl5PyMuCcrDD+77C3FJgOxnsg1FFus1knPZmYv1bSgSqY+cy86UbEORSY1+KRDSePaL7cxed5DLOjbm0UvbE/T4cHp8ekcTtSwjI/GUz9XELWzLgXaqqmarqmqhYlDs+b9aZj5wfeXXo4EvHQ6HVvn4OFVV41RVzQbaAT/VQCbRQGgauDx+BnZqUq31BnXKxOXxx/QLASGqStMg6HFBhyHVW7HDUIIemSFKiEjkXCZE7ZNzmRB1x+MP8ud5G5m97iDXn53F3y9TMRtronUgGroz3goqxzS6G/gM2AxMdzgcG1VVfVxV1WGVi70JpKuquh34A/Bg5bobgenAJuBT4HcOh0NGuYsxWiDIgI7Ve9F9a79sQn7ZVISoqvKAEa3vPdVaR+t7D26/DPArRFVogSC39sup1jpyLhOiesoDRjR742qtI+cyIaqnyO3jzhnrWLqzkD9f0pa7+2WjyCBiolKN3MDocDg+Bj7+1WN/PeFrDzDmFOs+BTxVEzlEw7R06xH+b94mrGYDHn/kKcbH9s6iVbqdEqerDtIJER08Hj/2tBzocU3Vpj+2JoPRgtdb9dtxhIhlHo+f1ul2xvRuwYwV+6q0zvy1B7iuR1NAXpgLEZEWwvDFYyirX4PkLCjeG3mdHtdCWjZep7f28wkRBfY5y7l39gYOlXr517BO9G/XSO9Iop6R69CErhZtOcw9s9bTONHCZ/ddwNjeWWGXH9s7i6dGdMFd5pFLkYWoBk2D4rIA2tDnK15Qh9N2AJoWQnv7MgxHN9dJPiEaOk0Dd5mHf4zoGvFcNrpnC0b2bM5LX27nyUXbCITkhCZEWEEviYvvwbb6NbzdbkC7e0Xkc1mPa9GGTqw498kuJkREG/NLuWnyGorL/bwyuqs0j8RJKVoDPKL6/UFNBjhr2DRN44MV+3jxmzx6tEjmueGdSE+0YkuwsqvAxaRv81i0KZ9ST4BEq4lBnTK5tV82rdLtuMs8+OWS/+NkwL/wpD6/ZDYbSU4wQeFOlGUvw5YFFTPUxCVBh6EVt7mlZVOatw773KtR/C5KLpuEv8V5ekfXjWxDkUmNfmY2G6t0LnOVlvPy1zt584c9nJedxj+GdsRmid3bbGQbCi+W66N4i0n65BYs+5dR1vchynvchdliqtK5rLgsIK8ZK8XyNlRVsVyjpTsLeOijzaTZzLwwqiut02Q2uliWkZG4Euh9suekgSTqXDCkMXHJDqatPsCA9hn87TKVOFPFxXCKAnFxZgxmI3ar+fg6Lo+fkD+I1yuDjf5aLJ/sqkLq81vH9jObOYjRaj/+eNDjwu03Ht/PDKUHSF5wLUbnTkov+Q/e9iN0TK0f2YYikxr9UnXOZbPXHeSZz7ehNk5g4ogupNstOqXWl2xD4cVqfQxlB0j+qPI8dPFzeNVRx5+r6rlMVIjVbag6YrVG89Yf5J+Lt9EuI4GJI7vQKEbPQ+Jn4RpINTIGkhBV5fEHeewTB19uO8qEXs2598IcDCcMyqZpFeNI4PFTXuYhOdkW89OuClHTju1nHg8oZaW/2s9+HocslNgM58jZJH18M0mL76Gs7CDlPe6seNUuhDil6pzLRnZrSobdwsMLNnPzlDW8MLILreSTXyEwFmwhecG1KN5Sioe+jz/r/F88X9VzmRDi5DRN441lu3lj2R76tE7l6Ss6YrdIe0CEJ2MgiTpTXO7n7pnr+WrbUe7vn8P9/dv8onn0a8deaEvzSIjaE2k/0+KSKR72IZ62w0hY9g8Svn0UQnI7gBBVVZVzWb826bw2thtuX5Cbp6xh/YGSugknRD1l3vcdKbNHghbCOXL2b5pHvyavGYWonkAwxJOLtvLGsj1c0bkJE6/sLM0jUSXSQBJ1Yn9xOTdPWcOmQ6U8NbQjE3q10DuSEKKqjHGUDnoZd+7txK9/h6TPbodAud6phIgqnZsm8eb4XJKsJu6csY6vtxfoHUkIXcRtm0fyR9cSsmfiHDWfYKNOekcSIqq4fUH+MHcj8zcc4pY+LXn00vaYjNIWEFUjW4qodZsPVYzoX+j28/LorgxUM/SOJISoLsWA67xHKTv/b1h2fkbKvPEoniK9UwkRVbJS43lzfC5tG9n58/yNzFxzQO9IQtQdTSN+9WskLfod/sweOEfOJpTYXO9UQkSVApePO6av5afdRTw8sB23n9caRYYmENUgDSRRq77LK+T2aWuxGA1MGt+dni1S9I4khDgD5d1voeTSVzEdWU/KrCsxlOzRO5IQUSXVZuHVsd04NzuNZ77Yzivf5tEQJzwRolpCQexLHyPh+yfxtL2C4is+RLOm6J1KiKiyq9DNTVPWkFfg5rkrOzOiW1O9I4kGSBpIotbMW3+QP87ZQMtUG29PyCUn3R55JSFEvedrO5TiYZMxlB8ldeZwTEfW6x1JiKgSbzby7PDOjOzWlHd+2svfPnXgD8qgwCJKBcpJWnQntnVv4e5+K6WDXgGTVe9UQkSVdQdKuGXKGsp9QV67qjvn56TrHUk0UNJAEjVO0zRe/34XTy7axlmtUvnfVd1olBCndywhRA3yNzsH58i5aEYLyXNGY96zRO9IQkQVk0HhwQFtufO81ny86TD3zd5AmTegdywhapTiKSJl/gQsOz6h7LzHcJ3/GCjy9kSImrRk21HumrGOJKuJtybk0jkzUe9IogGTI7SoUYFgiCc+kxH9hYgFwbR2OEfPI5TUiuQF1xO3ebrekYSIKoqicFOfljw2uD0r9xVz27S1HCnz6h1LiBphKNlLyuwRmA6tpeTSVynPvVXvSEJEnRlrDvDAR5tol2HnzfG5tEiJ1zuSaOCkgSRqjMsX4P65G/lo4yFu7Ssj+gsRC0L2TJwjZ+FvcR5JX/4B2/LnZR5lIWrY0M6ZPD+iM/udHm6avIadBS69IwlxRkxHNpAyazgG9xGKh0/G13ao3pGEiCohTeOlb/L41xfbOS87jVfHdCPVZtE7logC8u5e1IijZV5un7aO5buL+Mugdtx2rozoL0Ss0CyJFA95B486GvtPz5Gw5AEIya02QtSkPq3TeP2q7vhDGrdMWcvqfcV6RxLitJj3fE3ynFFgMOEcMRt/sz56RxIiqviDIR77xMF7y/cysltT/jW8M1azUe9YIkpIA0mcsbyCihH99xS5+feILgzvKiP6CxFzjBZKL5mIq9c9xG+aTNLHN4PfrXcqIaKK2iSBt8bnkmYzc/fMdXyx9YjekYSolrgtM0heeD2hpJY4R80jmK7qHUmIqFLmDXDv7A18uvkwd53fmgcHtMVkkA/1Rc2RBpI4I6v3FXPL1DV4AyH+d1V3zstO0zuSEEIvioK7zwOUXvgPLHu+ImXuGBT3Ub1TCRFVmiVbmTQ+lw5NEnnoo81MXrlP70hCRKZp2Fa8RNIX9+Nv1gfniFmEEuQDRyFq0pEyL7dNW8uqfcU8Nrg9N57TUu4IETVOGkjitH3uOMLvZq4jNd7MWxNy6dhERvQXQoCny3WUDH4DU6GD1FnDMTjz9I4kRFRJiTfzyuiuXNg2nYlLdjJxyQ5CMvaYqK9CARK+fhj7j8/gaT+C4qHvocUl6Z1KiKiys8DFjZPXsN/p4fkRnRnaOVPvSCJKSQNJnJbJK/fx0ILNdGqSyKTxuTRPlhH9hRA/8+VcinP4NBRfCamzhmPKX6V3JCGiitVs5OkrOnFVj2ZMXrmfvyzcgi8Q0juWEL/kd5P0ya3Eb3wfd8/fUTrgBTDKQL5C1KSVe53cMmUtgZDG61d1p09ruSNE1B5pIIlqCYY0/v3VDiYu2cnF7Rrx8uiupMSb9Y4lhKiHApm9cI6ah2ZJIGXeWCx5i/WOJERUMRoU/nhRG35/QTaLHUe4Z9Z6Sjx+vWMJAYDBdYiUOaOx7P6C0guewtX3IVDkrYcQNenjTYe4e+Z6GtktvDU+F7VJgt6RRJSTo7ioMo8/yMMLNjN11X7G92zOP4Z2lBH9hRBhBVNyKBo1j0CaStInN2Nd/67ekYSIKoqicO1ZWTxxeQfWHSjhlqlrOVDs0TuWiHHGAgcpM4dhKtpGyeVv4el6vd6RhIgqmqYxadluHvvEQffmSUwa351myVa9Y4kYIA0kUSVOt5+7Z67nq21Hub9/Dn+4qA1GGdFfCFEFmi0D5/Dp+FpdTOI3j2D/7gnQ5FYbIWrS4I6NeXl0V46W+bhx8mo25pfqHUnEKPO+70iZPQJCfpwjZuFrPUDvSEJElUAwxBOfbeV/3+/m8k6NeWlUV5KsckeIqBvSQBIR7Skq56Ypq9l8qJR/DO3IhF4t9I4khGhoLHZKLnuT8q43YFvzP5I+vR385XqnEiKq9MpK4c3xuVhNBm6ftpavt8ssiKJuxW2ZSfJH1xBKaIpz1HwCjbvpHUmIqFLmDXDv7A18tPEQt/Rpyd8Gq5iN8pZe1B3Z2kRYa/YVc9Pk1ZR4Avx3TDcGqBl6RxJCNFQGI2X9nqDs/L9h2fkpKfPGorjlDa4QNSk73cZbE3rQtpGdP83bxJRV+/WOJGKBpmH76T8kfXEf/qZn4xw5m1CSfOAoRE3KL/Fwy9Q1rNxXzF8vbc/t57VGUeSOEFG3TGeysqqqacA0oDWwCxjrcDiKfrVMLvAqkAQEgaccDse0yufeAS4EiisXv8HhcKw5k0yi5izacpi/feqgaZKV50d0IStVZloTQpwhRaG8+y0EE5uTtPgeUmcNo3joewRT2+qdTIiokW638NrYbjz68Rb+89UO9jvLub+/3HouaknQR+JXf8bqmImnw1hK+z8tM60JUcO2HCrl/jkbKfcHeXFkF85ulap3JBGjzvQKpAeBLxwORzvgi8rvf80NXOdwODoDg4HnVVVNOeH5PzkcjtzK/9acYR5RAzRN4+0f9/DIwi10yUzkzfG50jwSQtQoX85lOK+cgeJ3kzJrOOb9y/SOJERUsZqNPH1FJyb0as601Qf407yKNx5C1CTFW0zyR9dgdczEdfYfKb3439I8EqKGLd1ZwG3T1mI0KEwanyvNI6GrM20gDQeOTanzLnDlrxdwOBxbHQ7HtsqvDwCHAbkPqp4KBEM8tWgb/126i0s7ZPDy6G6kxMugbEKImhdo0oOi0fMJ2RqTPH8CcY5ZekcSIqoYDQr392/Dny5uy3d5hdw+bS1Hy7x6xxJRwlCyj5RZIzAfXE7JJc/jPut+kNtphKhRM9cc4I9zN9Iq1cbbE3Jp28iudyQR4860gdTE4XAcrPw6H2gSbmFVVc8GLMCOEx5+SlXVdaqqTlRVNe4M84gzcGxQtnkb8rm5T0ueuLwDFpMMkyWEqD2hpJY4R87B3/Qskj6/F9vyiaBpescSIqqM7dGM54Z3Zlehmxsnr2H7UZfekUQDZzq8lpRZwzC48im+4gO8HUbrHUmIqBLSNF78eifPfLGdc7PT+N9V3clIkLfKQn+KFuGFuqqqnwOZJ3nqEeBdh8ORcsKyRQ6H46TX1Kmq2hRYAlzvcDh+OOGxfCqaSq8DOxwOx+ORQodCIS0YlDcYNWm/s5xb319J3lEXTwzvzOie9WPgQ6PRQDAo032HIzUKT+oTWb2oUdCHceF9GNZPJdRtPMHLJ9ab2yDqRX3qOalRePWlPhsPFHPbB6tw+4K8PD6X89o00jvScfWlRvVVfaqPsvVTjHNvAVs6gaumQUYHvSMB9atG9ZHUJ7L6UiOPP8ifZ63nk435TDg7i0cv74hJZloTdchsNq4Eep/suYgNpHBUVXUA/R0Ox8FjDSKHw6GeZLkkKppH/3A4HDNP8bP6A//ncDiGRvq9fn9Qczrdp51b/NLmykHZvIEgz1zRqV7dV5uSYkP+1uFJjcKT+kRWb2qkadhWPI/9p3/ja34uJZe9gRaXrHeq+lOfekxqFF59qk9+iYf752wkr9DNwwPaMazryT4jrHv1qUb1UX2pj3X9OyR8+1cCjbpQPOQdNHtjvSMdV19qVF9JfSKrDzVyuv38cd5G1h0o4fcXZHNN7xYy05qocxkZiadsIJ1pK3M+cH3l19cD8369gKqqFmAO8N6vm0eVTSdUVVWoGD9pwxnmEdX09fYCbpu6FotR4U0ZlE0IoSdFwX3W/ZQMeB7zweWkzLoSQ8levVMJEVUyk6y8Ma47Z2Wl8MSirby6NI8z+TBRxIhQEPvSv5H4zV/wtboE54iZ9ap5JEQ02FXo5sYpq9lyqJR/Du3ItWdlSfNI1Dtn2kB6Ghioquo2YEDl96iq2ltV1UmVy4wFLgBuUFV1TeV/uZXPfaiq6npgPdAIePIM84hqmLpqP3+at5GcRnbemtCDnHQZlE0IoT+vOpriYR9icB8mdeYVmA6t1juSEFElIc7ExBGdGd41k7d+3MujH2/BF9D/tg1RPym+MpI+uRnb2km4u91EyWWTwGzTO5YQUWX5niJumrwGlzfIf8d0Y4Aqc06J+umMbmHTi9zCdmaCIY3nv97J1FX76d82nScu74DVbNQ71knVh0tJ6zupUXhSn8jqa42MRdtJXnAdBvdhSga+hC/nMl1y1Nf61CdSo/Dqa300TePdn/byytJd5DZP4tnhnXWbebW+1qi+0Ks+htIDJC+8HmPhVsr6PY6n6/WRV9KJbEPhSX0i06tGc9cd5OkvttMyNZ6JIzrTPDm+zjMIcaLavIVNNDBl3gB/nLuRqav2M6FXc56+olO9bR4JIWJbMLUtRaPmE0jvSNIntxG/6hWZoU2IGqQoCjec05KnhnRgU34pN05eza4CeYMpKpgOrSFl5lAMpfsoHvpuvW4eCdEQBUMaL3y9k6cWb+Oslim8NT5Xmkei3pMGUgw5UOzhlqlr+GFXIQ8NaMv9/dtgNMh9tUKI+kuzNcJ55XS87YaRsOyfJH75Bwh69Y4lRFQZ1KExr47tjtsX5MYpq/lxV5HekYTOLNsXkDJnFBjjcI6ci79lf70jCRFV3L4gD8zfxAcr9jEmtxkTR3QhIc6kdywhIpIGUoxYu7+YGz5czeFSHy+M6srI7s30jiSEEFVjiqd04Mu4zvoD1i0zSJ43AaW8UO9UQkSVbs2SeOfqHjRJjOPe2euZseaA3pGEHjQN24qXSP7sDgIZXSka/RHB9N9MsCyEOAOHSr3cOnUN3+4s4P8uasOfL2mLST7UFw2ENJBiwCebD3HnjHUkxBl5a0Iu58hMa0KIhkZRcJ/9B0oGvYL58BpSZw7FWLhV71RCRJWmSVbeHJ9L3+w0/vXFdp77cjuBkNw2GjOCXhK//AP2H5/B0+5KnMOnotka6Z1KiKiy+VDF7cL7iz3858ouXNWzud6RhKgWaSBFsZCm8erSPP76sYNuzZJ4a0IPWqfJrBlCiIbL2244zitnoPjLSZk1HPOeJXpHEiKq2C0mnhvemQm9mjNt9QHun7OBMm9A71iilinlhSTPm4B1ywxcZ/+R0oEvgcmqdywhospX245y29S1mAwKk8blcl5Omt6RhKg2aSBFKY8/yMMLNvPWj3sZ3jWTl0Z11W1mFSGEqEmBzJ4UjVlAKDGL5AXXYV33tt6RhIgqRoPC/f3b8PDAdizf4+SmKWvY5yzXO5aoJcaiHaTOvALz4TWUDHoF91n3gyK30whRUzRN472f9vLA/E20zbDz9oQetM2w6x1LiNMiDaQodKTMy23T1vLl1qPcd2EOjwxsh9kof2ohRPQIJTanaOQcfK0GkPjtoyR8/QiE5CoJIWrSiG5NeXlUVwpcPm6cvIbV+4r1jiRqmHnfd6TMGobid1VOWDBc70hCRBV/MMSTi7by0rd5DFAzeHVMN9LtFr1jCXHapKsQZbYcKuWGD1ezu7Cc567szNW9W6DIp0hCiGhksVNy2Ru4e9xB/IZ3SV5wPYpX3uAKUZN6t0zh7Qk9SLKauGvGOhZszNc7kqgh1vXvkjx/AiF7JkWjPyKQ2UvvSEJElSK3j9/NXM/8DYe4pU9LnhzSAavZqHcsIc6INJCiyJfbjnLr1LUYFIVJ47tzQZt0vSMJIUTtMhhxnfsXSi96DvP+70iZNRxD8S69UwkRVVqmxvP2hFxyWyTz90+38vK3eYQ0GVy7wQr6SVjyEInfPIKvZX+co+YSSsrSO5UQUWXbkTKu/3A1m/JLefLyDtx+XmsM8qG+iALSQIoCmqbx9o97eGD+Jtpl2Hnn6h60y0jQO5YQQtQZT6dxFA+bjMF9lNQZQzHv/17vSEJElSSrmZdGdmFEt0zerRzLw+WT20YbGqW8kOT544nf+D7unr+j5PK30CyJescSIqp8te0oN09ZQzCk8fpV3bm0Y2O9IwlRY6SB1MB5/EEeWbiF/y7dxaUdMnh1bHe5r1YIEZP8zc+laPRHhGyNSJ43Huv6d0CukhCixpiMBh4a0I4/XNSGb3YUcLMMrt2gGI9uInXGEMyHVlMy8CVcfR8Cg9xOI0RN0TSNSct28+f5m2jTyM67V/egU6Y0aEV0kQZSA5Zf4uGWqWv53HGEe/pl88TlHYgzyZ9UCBG7QinZOEfNx9eyP4nf/IWEJQ9A0Kd3LCGihqIojO/ZnBdHduVImY8bPlzNT7uL9I4lIrDs/JTUWVdCyIdzxCy87UfoHUmIqFJeOQP2/77fzZBOjXltbHcaJcTpHUuIGifdhgZq9b5irvtgNfuc5Uwc0YXrzs6SwbKFEALQ4pIoufwtXL3uIX7TZFLmXYXiPqJ3LCGiyjmtU3lnQg/S7BZ+P2s9U1ftR5Mr/uofTcO24gWSP7mFQFp7nGMWEmiSq3cqIaJKfomHW6as4cttR7n3whweG6zKh/oiasmW3QDNXnuAO2esI9Fq4p0JPTgvJ03vSEIIUb8YjLj7PEDJoP9iOrKe1BmXYzq8Tu9UQkSVrMrBtc/PSeffX+3gyUVb8QVCescSx/jLSfzsTuw/PotHHYVzxExC9ky9UwkRVdbur/hQf3+xh/+M6MI1MgO2iHLSQGpAAsEQT3++jX9+vp2zW6bwzoQetE636R1LCCHqLW+7YThHzgUMpMweQdzWuTonEiK62C0m/jW8Ezf3acn8DYe4Y/pajpZ59Y4V8wylByqOeTsWUtb3EUoveR5MVr1jCRFV5q/P547pJ3yony0f6ovoJw2kBqLI7eOumeuZtfYg153VgokjupBoNekdSwgh6r1ARheKxizE3ziXpMV3Y1/2DwgF9Y4lRNQwKAp3nNeap6/oyLYjLq7/cDUb80v1jhWzTAdXkDrjcowluykZ8g7lPe8EuSJCiBoTCGn8+6sdPLFoK72yknl7Qq58qC9ihjSQGgDH4TKu/3A1m/JLefxylXsuyMFokBcCQghRVZqtEcXDp1De+Vpsq/5L0sIbULzFescSIqpc0j6DN8fnYjIo3DZ1DR9vOqR3pNiiaVg3fEDK3DGELAkVEwq0vkTvVEJElSK3j3sqx32b0Ks5z4/sSpLVrHcsIeqMNJDquc8dR7hlyhqCIY3Xr+rOZR2b6B1JCCEaJqOFsv7/pPTCf2LZ9y0pM6/AWLRD71RCRJX2jRN49+qedG2WxGOfOHh+yU4CIRlcu9YFPCR89ScSv34Qf4vzcI5eQDCtnd6phIgqmw+Vct0Hq1m3v5jHBrfn/v5tMMmH+iLGSAOpngqENF74eicPLdhMu4wE3r2mJ50yE/WOJYQQDZ6ny7UUD5+KwVtMysyhWPIWn3LZY3d9yN0fQlRdis3My6O6Mja3GR+u3Md9s9fjLPefcnnZz8KLVB9D6QFS5owifvNUXL1+T/GQd9GsKXWWT4hoEGk/+2hDPrdMWQPApPG5DO0sA9KL2KQ0xClX/f6g5nS69Y5Ra4rcPh5esJkVe4sZ3b0pf7ioDWZjbPb6UlJsRPPfuiZIjcKT+kQWqzUylOwj6dNbMR9Zj6v3vbjP+gMYjCgKWK1m4k1BjFb78eWDHhflASMej58GeOqsVbG6DVVVLNdn3vqDPPPFdhrZLTwzrBMdm1R8GHZsP1NMRuwn3P7h8vjRAsGY38+qehwy719G0md3QMBL6YDn8eUM1jG1vmJ5P6sKqc9vVeU45AuEmLhkJzPWHKB3yxT+MaQDqTaLjqmFqH0ZGYkrgd4ne05GYa5nNh4s4c/zN+Es9/PXS9tzRRfpbgshRG0IJbXAOXI2CV//BfuKFzAfXkP55a+S1CgDCneiLHsJtiwEbwnEJWHsMAR733uwp+VQXBbA75eBuIWIZHjXprTNSOCB+Zu4ZcoaHrikHaN6NseWYGVXgYs3vnWweNMhSj0BEq0mBnZqwq39cmidbsdd5onJ/cxsNpKcYAp/HErNpnzpa8R/83eCya0pGfEmwdS2ekcXosEwm40Rj0N2i4nfTV3Jmn3FXNO7Bb/rly23rImYJ1cg1SNz1h3k2S8rPqX717BOdGgit6zJpyWRSY3Ck/pEFvM10jSsmz4k4du/QmImSpMu4Pj41Mv3uBZt6ESKS/0x+eb2ZGJ+G4pA6lNxdfUjC7ewfI+Tq87Kwh8MMXvV/lMuP7Z3Fk+N6IKrNLaaSGazkeREM8qC+2D1B6deMDUbivLwtRlMycUT0SzymlH2s/CkPj8zm43YE608PGc9M1bsO+VyVrMBBYXHLlO5pF2jOkwohL5q7QokVVXTgGlAa2AXMNbhcBSdZLkgsL7y2z0Oh2NY5ePZwFQgHVgJXOtwOHxnkqkh8gZCPPvlduatz6dPq1SeGNKBlHgZzV8IIeqEouDtcg0J2T1QPhgRvnkEsPp9FCB5yH8ocAZj+jYbIaoq1WbhpdFdeXP5ft74dmfE5aev2AvAk1d2ocTpion9TFEgOcEUuXkEUJQHTbtjHvcuFPshBuojRE1QFLAlRG4eAXj8IQZ1asLIs1vFzHFIiEjOdGCdB4EvHA5HO+CLyu9PptzhcORW/jfshMefASY6HI62QBFw8xnmaXDySzzcNm0t89bnc+M5WTw/sos0j4QQoo5ZrWawJoO/ip/Orn4fCncSFyfHayGqKsFmYUyvFlVefvqKvewqcMXMfma1mqFgR+Tm0TEH10LRrpipjxA1wWo1s+uoK2Lz6JhFmw7F1HFIiEjOtIE0HHi38ut3gSuruqKqqgpwMTDzdNaPBsv3FHHtB6vZXejmueGduOv8bIxyX60QQtS5eFMQ5YeXq7WOsuxlbObYubVGiDOlmIy8sTTy1UcnmvRtHgazsZYS1S9yHBKi9slxSIgzc6YNpCYOh+Ng5df5QJNTLGdVVXWFqqo/qKp6ZeVj6YDT4XAEKr/fBzQ/wzwNRnG5n3tmbSDVZuadq3twYVu5r1YIIfSgKFTMcrRlYfVW3LIAo9UuU48LUQWKAnarmcWbDlVrvUWb8rFbzVG/n8lxSIjaJ8chIc5cxDGQVFX9HDjZVGCPnPiNw+HQVFU91Z2hrRwOx35VVXOAL1VVXQ8UVzttJaNRISXFdrqr1wvJyRr/u7onvVqlkhAnk+GditFoaPB/69omNQpP6hOZ1KiSt+S0lk9OltrJNhSe1OdnpZ5A5IVOsnzM7GdyHDptsp+FJ/X5mRyHhDh9ETsXDodjwKmeU1X1kKqqTR0Ox0FVVZsCh0/xM/ZX/rtTVdUlQA9gFpCiqqqp8iqkFsCpp+M4QTCoRcUsAt0b2wmU+3CWx9y44VUmM0ZEJjUKT+oTWazXSFGgUaNEiEuC8t/MA3FqcUkAFBe7Y35gzVjfhiKR+vy8nyVaTTjd/iqvl2iteKka7fuZHIfOnOxn4Ul95DgkRFVlZJx6Zs8zvYVtPnB95dfXA/N+vYCqqqmqqsZVft0IOA/Y5HA4NOArYHS49YUQQojapGkQ9Ligw5DqrdhhKEGPzMoiRFVoGrg8fgZ2OtVoByc3qFMmLo8/6vez48ehtgOrt6Ich4SosmPHoa7Nk6u1Xqwch4SoijNtID0NDFRVdRswoPJ7VFXtrarqpMplOgIrVFVdS0XD6GmHw7Gp8rkHgD+oqrqdijGR3jzDPEIIIUS1lQeMaH3vqdY6WlwSbp8MiCBEVWmBILf2y6nWOgM6Nibkj41Bor37N6LtXlqtdbS+9+D2y+C+QlRFMKTx8hfbWLrtaLXWu7Vfdswch4SI5IwG33E4HAXAJSd5fAVwS+XX3wNdT7H+TuDsM8kghBBCnCmPx489LQd6XFO1KbRTWqH88ApxBzfjHfACmjW19kMK0cB5PH5ap9sZ07tFlabQtpoM/H7qav58cVuu6HKy4TijhKZh3fgB8Uv/BrZ0aH8pbP0s8no9roW0bLxOb61HFKKhK3D5eOyTLfy428mVuc1RFI05qw9EXG9s7yxapdspcbrqIKUQ9d+ZXoEkhBBCNHiaBsVlAbShz1e8KQunx7Vod6/AfdE/Me9dSuq0QZgOrqiTnEI0ZJoG7jIP/xjRlbG9s8IuO7Z3Fl/+X3+6Nk3i8c+28rdPHZRH4RUAireYpM/uIPHrh/A360PJhEVoV31YtePQ0IkVxy25rUaIsH7cVcSE91ayel8xjwxsx2OXtuNfo7tX6Tj01IguuMs8sp8JUUnRGuDe4PcHtVgfBC5WyIB/kUmNwpP6RCY1+pnZbCQ5wQSFO1GWvQxbFlTMchSXBB2GVtzmlpZNcVkAvz+I6fA6kj67A0PZAVx9HqI891ZQYu+zGdmGwpP6/JLZbMSWYGVXgYtJ3+axaFM+pZ4AiVYTgzplcmu/bFql23GXefB4A0xatps3f9hD63QbT1/RkZx0u97/CzXClL+SpEW/w+DKx3XOnynvcQcohmofh0QF2c/Ci8X6BIIhXvt+N+/9tJfWaTb+MbQjbTMqjh/VOQ7JfiZiTUZG4kqg98mekwaSqNdi8WRXXVKj8KQ+kUmNfklRIC7OjM0cxGj9+Y1q0OPC7Tfi9f5yIE3FW0zil/9H3M5P8LXsT8klE9FsGTok149sQ+FJfX7r2H5mMBuxW83HH3d5/IT8wd/sZz/uKuLRj7fg9gf5w0VtGNE1E0VpoGOQaSHiV/0X+4/PEkpoRsmgVwhk9vzFItU9DgnZzyKJtfocKPbwl4WbWX+wlCu7ZvLHi9pgNf9yvLDqHoeEiBXSQBINVqyd7E6H1Cg8qU9kUqNTUxRITrZFnrpX07BufJ+EpX9HsyRScslE/K0uqrOcepNtKDypT3hV3c+Olnn526cOftzt5KJ2jXhkYDuS482nXqEeUlyHSfr8Xiz7vsXT9grK+j+NFhd+RqgqH4dinOxn4cVSfT53HOGpxVvRNHh4YDsGdWgccR3Zz4T4WbgGUuxdZy+EEEJU0bEXkRFfTCoKni7XUTRmIaH4dFIWXIt96d8hKIPbChFJVfezRglxvDiqK7+/IJtvdxQw4b2VrNzrrPV8NcW8+yvSpg3EnL+c0ov+Remg/0ZsHkE1jkNCxDiPP8g/Fm/loQWbaZVq48PrelapeQSynwlRVdJAEkIIIWpIML0DRWMWUN71emxr3yBl5jCMRdv1jiVE1DAoCteelcVbE3Kxmo3cOX0dry7NIxAM6R3t1II+7N8/ScqCawnFN6JozMd4Ok2ouORBCFEjdhx1cf2Hq5mzLp/rzmrBpHHdaZ4cr3csIaKONJCEEEKImmSKp+yCpyi+/G2MZQdInX4Z1k2T5WNNIWpQxyaJvH9NT4Z2bsJbP+7ltmlr2V9crnes3zAWbiNl1nBsq1+jvPO1FI1ZQDCtvd6xhIgaIU1j6qr9XP/hapzlfl4a1YV7LsjBZJS3uULUBtmzhBBCiFrgyx5I0bjF+DN7kfjVn0n67A4Uj1PvWEJEDZvFyF8Hqzw1pAN5hW6ufm8Vn24+rHesCpqGdf07pE4fjLF0H8WXvUFZ/3+CSa6IEKKmHCnz8vtZ6/n3VzvonZXCh9f1ok/rNL1jCRHVTHoHEEIIIaJVyJ5J8bDJxK9+FfuPz5Kav5LSSybiz+qndzQhosagDo3p0jSJRz/ewqMfb2HpzgL+fElbkqz6DLBtcB0i8cs/YtmzBF/L/pRe/G9C9ia6ZBEiWn2x9Qj/XLwNTyDEgwPaMrJb04Y7M6MQDYhcgSSEEELUJsVAec/f4Rw1D82SQMr88di/eRT89e92GyEaqmbJVv53VXduO7cVn289yvh3V/LjrqI6z2HZ8TGpUwdg3r+M0gueonjo+9I8EqIGlXkD/O2TLTz40WaaJVv54NqejOreTJpHQtQRaSAJIYQQdSDQuDtFYz/B3e1mbOvfJnX6pZgOrdY7lhBRw2RQuLVvK94an4vdYuLuWev51xfbKfcHa/13K74yEr74I8mf3kYwMYuiqz7D0/V6GShbiBq0el8xV7+3kk82H+bmPi15a3wurdNsescSIqZIA0kIIYSoK6Z4XP3+jnP4NJSAh5RZV2L78TkI+vVOJkTU6JSZyHvX9GBCr+bMWHOAa95fxfoDJbX2+8wHfiR12iCsjhm4ev0e56i5BFPb1trvEyLW+AIhXv42j9unrcVgUHhjXC53nNdaBsoWQgey1wkhhBB1zN/iPIrGLcbbfgT2Fc+TMmsYxsKtescSImpYzUbu79+GV8d0wxcIccvUNbz63S78wVDN/RJ/OfZvHyN5zmgAnCNm4e7zZzBaau53CBHjNuaXcs0Hq3j3p70M65LJB9f2pFuzJL1jCRGzpIEkhBBC6ECLS6Z0wPMUD34dY+l+UqdfRvya1yFU+7fbCBErerdMYcr1vbi8UxPe+mEPN05ew/YjrjP+uaYDP5E6bSC2dW/i6XodhVctJtD0rBpILISAiquO/rs0j5snr8blDfD8iC785dL22C0yB5QQepIGkhBCCKEjX5vLKRz3Ob6sC0j47nFSZo+Qq5GEqEEJcSYeG6zy7LBOHC71cu0Hq3j9+9O8Gslfjn3p30iZMwolFMQ5fBplFzwFFnvNBxciRm3KL+XaD1bx9o97ubxTE6Ze35vzctL0jiWEAKSFK4QQQuhMszem5PK3iNs2l4Rv/0rqtMG4z7ofd487wKjPVORCRJv+7RqR2zyZfy/ZwRvL9vDVtgIevbQ9nTITq7S+6eByEr/4A6biPMq7XE9Z34elcSREDfIFQkz6YTfv/bSXNLuF50d0kcaREPWMXIEkhBBC1AeKgrf9CArHf4U351LsPz5DysyhGI9s1DuZEFEjxWbmics78J8rO1Ps8XPj5NW89E0ennAztfnLsS/9OymzR6KEAhVXHV0oVx0JUZM25Zdy3YcVVx1d1qkJ0+SqIyHqJbkCSQghhKhHNFsjSi99FW/bK0j8+hFSZw7B3eMu3GfdC8Y4veMJERX6tUknt3kyL3yzk/eW72XJ9qP89dL2dG+e/IvlzHu/IXHJQxhLdstVR0LUgnJ/kNe+28XUVftJt1uYOKIz5+ek6x1LCHEK0kASQggh6iFfm8spbN6XhKV/x77yReJ2fkrpxc8SyOyldzQhokKi1cRfBrVnYPsMnlq8lVunrmVsj2bcdX429mAxCd/9HatjFoGUHJxXTsff/Fy9IwsRVb7PK+Tpz7dxsMTLiG6Z3NMvh0SrvD0Voj6TPVQIIYSopzRrKqUDnsfbbhgJSx4gZdaVeDpfg6vvg2hxyZF/gBAionNapzL1+t78d2ke01fvx+qYxcPG97EEy3D1vhd3r3vAZNU7phBRo9Dt4z9f7eCzLUdonRbP61d1p0cLOacJ0RBIA0kIIYSo53ytLqZo/FfYfnyW+PVvE7fzU8rOfwxvu+GgKHrHE6LBs1mM/LmXmT8XvEL64e9ZGWjHzMxnmNDxEjKleSREjdA0jQUbD/HC1ztx+YLc2rclN5zdEotJhuUVoqGQvVUIIYRoADRLAq5+f8c5ZiHBxGYkLb6b5I+uxuDM0zuaEA1b0Ef8ypdJm3oJqUXrKO73FF+f8w5zDiQz5u0VvL98L4FgSO+UQjRouwrc3DVzPY9/tpXsdBuTr+vFbee2luaREA3MGV2BpP5/e/cdH1d15n/8M00aadRl2ZYLklw4brhhDAZMMS2ACZAACQmkAUmWDZvkF7IpbHoj2QRSSCHAZhM2CaGEQOjVwQRMbDDF2D7uvUq2ZLWRptzfH3dkZGPNSFYZaeb7fr30GuuWmUePz70z89xzzjWmDPgLUA1sAq6w1u4/bJszgVs7LZoEfNBa+zdjzP8CpwMNiXUfs9a+3puYREREMlm04jjq3/8wwRV/ILTkh5TdczYtc/6Dllmf1iTbIj0U2PoCBS98DX/9etrGvYem+d8hXlDJ1cDZk0bwo2fX8fMXNvL4qj185eyJHDeqKN0hiwwpLe0x7lqyhT+9uo1gwMtXzp7AJdMr8ar3rMiQ1NshbF8GnrXW3myM+XLi9y913sBa+zwwEw4WnNYBT3Xa5IvW2vt7GYeIiEj28PoIT/847ePPJ7T4m4Re+W9y7V9pmv9tIsecnu7oRAY9b+MOCv75LXLXP0q0uJqGhX+gvWrBIdtUFgW55ZKpLFpXx4+fW8c1f36di48byfWnVlOan5OmyEWGBsdxeHZNLbcuWs+epnYumjqCz5xWQ5mOHZEhrbcFpIuBMxL//j2wiMMKSIe5DHjcWtvSy9cVERHJevHQSBrf8xvaNj9HaPHXKfn7h2mrOY+mU75OvLgq3eGJDD6xdvJe/y2hZT8DHJpP/E9aZn6yy0myPR4PZ04cxtyqEn770mb+snwHz66p5VMnV/H+maPwe9WLQuRwm+pa+O/n1vGvLfUcWxHiBxdNYbp674lkhN4WkEZYa3cm/r0LGJFi+w8Ctxy27HvGmK8DzwJftta29TImERGRrNJetYD2MaeQ98adhJb+jLI/L6Bl5qdoOf4zEMhPd3gig8Ihw9VqzqPp1G8SLxrbrX1DOX4+f8Z4Lj5uJD95bj0/fn49f31zJzcuGM8Jx5T2c+QiQ8Phw9W+uGAC759RiU+FVpGM4XEcJ+kGxphngJFHWHUT8HtrbUmnbfdba4/4LmqMqQTeBEZZayOdlu0CcoDfAuuttd9OFXQ8HndiseRxS2bw+bzENHFlUspRcspPaspRckMuPwd24Hv+W3hX3IdTOIrY2d/GmXxpv96tbcjlaIApP6n1a45q1+B79ht41z2JUzqO2Lk/wJlwzlE/neM4PLNqD99/fDXb6lt5z9QRfPk9kxhdkteHQR9KbSg15Si5/sxPLO7w1+XbufWZtextauP9s0fzxXOOpbxgaM3LpzYk4goEfK8Cc460LmUBKRljjAXOsNbuTBSDFllrTRfbfhaYaq39ZBfrzwButNYuTPW6kUjMqa/XKLhsUFKSj/6vk1OOklN+UlOOkhuq+fHv+BcFi79OoHYF7aNOpPmUbxAdPr1fXmuo5migKD+p9UeOPOH95P/rFvLevhvHn0fL8TfQOv0TXQ5X66lwJMYfX93G717ZCsDVc8Zw9Qljyc/x9cnzd6Y2lJpylFx/5WfZlnpuXbSeNXubOa6yiM+fMW7ITjavNiTiqqgo7LKA1NshbA8DHwVuTjw+lGTbK4GvdF5gjKlMFJ88wCXAil7GIyIiIkB01FzqL3+U4Ko/E1ryI0rvu4DwxEtoPulL3R62IzIkxdrJe+v35C/7KZ72RsJTPkzz3C/g5A/r05cJBnxcc1IVF04Zwc9f2MidS7bw4Fu7+OTJVbx32kjNjyQZbfO+Fn7+wkZeWF9HZVEu37twEueYCjy6u5pIRuttAelm4F5jzDXAZuAKAGPMHODT1tprE79XA2OBfxy2/x+NMRWAB3gd+HQv4xEREZEOXh/hqVfRNuG95L/2K/LeuIPc9Y/ROv3jtBx/A06wJN0RivQdxyFn41OEXvou/oaNtI89naZTvkasfFK/vuzIoiDfXziZK2eP5mf/2MAPnl7LPa9u5zOn1TB/XJm+UEtGaWiNcNeSLdz7+g5yfV6uP7WaK2ePJhjo+553IjL49GoIW7poCFv2UFfS1JSj5JSf1JSj5DIpP96mHYRe+TG5q+/DyS2iZc7naD3uI+Dr3TwVmZSj/qD8pNbbHAV2LCH08s0Edi0jWjqR5lO+RvsxZ/br3F9H4jgOi9bVcdvijWzZ38rsMcV89vRxTBlZ2KvnVRtKTTlKrrf5aY3EuOe17fxh6VZa2mO8d9pIPn1KNeWhnD6MMr3UhkRc/TmETURERIaIeMEoGs+6hZYZ11Lw0vco+Oe3yHvrdzTP/QJtEy8Br64gy9Di37uC0JKbydmyiFhoBI2n30x48gfAF0hLPB6PhzMnDmP+uDIefGsXd7y0mY/+cTnnmgo+eXIVVWW6K6IMLZFYnAff3MldS7awryXC/HFlXH9qDRMqQukOTUTSQAUkERGRLBMbNoWG9/6RwJZ/EHr5+xQ981mir95G89wv0D7+AvB4u/1cHR08PB4Ygp2aZRA4mjbkq99A/iv/TXDd34nnltA07yZap38M/P13J7Se8Pu8XD5zFOdPHs7dS7fyp1e38+yavVwwZQTXzDuG0cXdj1PHmPTW0bShWNzhiVV7+O1Lm9hxoI3ZY4r50XurmTG6uP8CFZFBT0PYZFBTV9LUlKPklJ/UlKPkMj4/Tpyc9Y8R+tdP8O9fS7R8Cs0n3kh79TldDv/xeCAYDJDnj+ELvnMVOhZupjXqIxyO6ItuJxnfho7C0bYhb+N28pf9lOCqe8GXS8vM62id+Smc3MF916e65nb+sHQr97++g5gDlxw3ko+feAwjCo88fLQjPx6/j1Dwnd5UzeEITjSmY+wIdJwd6mjbkOM4/GNdHb/+5yY21LUwaXgB18+v5qSq0oyfz0ttSMSVbAibCkgyqOlEnppylJzyk5pylFzW5CceI3ftQ+QvvQV/wyYiw2fQfOIXiYw9/ZBCUiDgo7jAD/s24Hn5F7D6UWg7ALlFMOlCnHk3QNk4GpqiRCKxNP5Bg0fWtKFuOpo25D2whfxXbyO4+j7AQ+u0q9yJ4PMr0vvH9NCexjZ+98oW/vbWLrweeN+MUXx07liGdZpHJhDwkV8QZFNdM3cs3sDTK3fTGI5SGPRzzpQRXDd/HNXlIVqawjrGOtFx9o6jaUNxx+H5tbXctWQLa/c2c0xpHv92SjULjh2GN8MLRx3UhkRcKiDJkKUTeWrKUXLKT2rKUXJZl594lODq+8lf9lN8jduIjDyeluNvoL3qLAI5fooLA3ge+Rws/7+un2PW1TgLb6WhMaIvuGRhG0oiEPD1qA01bl1DYMnPCNr7weMjPOVKWmZfT7xw9IDF3B92Hghz18tbeOTtXfh9Xi6dXsmHjx/N2PIQocIgX33wLe5btq3L/a+YM5bvXTqN5kYVkTroOHMFAr4etaEDDa088fYu7lqyhQ11LVSV5vGJk47h3EnD8Xuzo3DUQW1IxKUCkgxZOpGnphwlp/ykphwll7X5ibUTXPUX8l/7Jb7GbUTLJ+Ob/zk8m/4Br/8p9f6zrsa58Bbq6tuyfqhN1rahw3g8UF6Si+fRzycvHnUoHYdTvwm8AVqnXkXrrE8TL6js9zgH0tb9rdz1yhaeWLUHD3DJrNE0tUV4YsXulPteMWcs371kGgfqm7P+GAMdZ+AeY0UlIW76W/LiUYcTa8rY09jGxtpmasrzufakYzjr2Ap8WVY46qA2JOJSAUmGLJ3IU1OOklN+UlOOksv6/MQi5K57iILXfoV335oe7epc/wpNedWEw5F+Cm5oyPo2lJCXFyDUvBHPr0/q9j7OjA/RPO8rtPpK+zGy9Nt5IMw9y3dw//IdtMfi3d7vqc+fRmV+IOuPMdBxBu4xtqMpwrk/faHb+4wbFuLfT6vh1OrSrBmq1hW1IRFXsgJS92+zIiIiItnHF6DNXIZz3XNQc1qPdvW8fBv5AQ2vEVeeP4ZnyW092sfj9ZFXUt5PEQ0elUVBblo4hfOmjejRfncu3og34OunqGSo8fh93PHihh7tc3xVKe+ZPirri0ci0j0qIImIiEhSHg/48gph11s923H1I/iCoa5u5iZZxOPBvdva6kd7tmOWtCGPB0LBAIvX1vZov6dW7iIUDGR8fiS1jjb09MrUwx87e3rVbrUhEek2FZBEREQkqYO3bm470LMdE9tn+q2fJTW1oeQ6/r7GcLRH+3Vsn+n5kdTUhkRkIKiAJCIiIkkdnC8xt6hnO+YUHLq/ZC0nHoM1TwE9/JKaaHOZ3oY6/r7CoL9H+wUDXhzHyfj8SGqO41Db1EbA17NjrKPNqQ2JSHeogCQiIiJJOQ7Ews0w6cKe7djWRPz+a/HtWIZuE5WdPC17yVv+G0rvng9/uhy8PSuQMGkhsXDm32XMcaA5HOGcKT2bA6m5LcaZP17E3Uu3Ud+iibSzkeM4vL6tga89upp5P3iWcKT7k7ADnDtlJM3hSMYfYyLSN3r4Li4iIiLZqDXqIzTvBjzduf16gjPlYjxrHqd0xX1EyyfROuXDtB17CU4ws++olfXiUXI2P09w1T3kbH4WTzxKpHIukdO+Su7oqXhuP7XbT+XMu4GWiA/o2ZfiociJxrhu/rhu3X69w3+eZ3j67V387B8b+NWLG1kwcRiXTq9k1phiTYqc4fY0tvHoyt088vZutuxvJT/g4/JZo1kwZSTX/mFZt5/nuvk1xCO62YGIdI8KSCIiIpJSOBwhVDYOZl0F3SkizboaLryFuj37yF3zN4Jv/5HCxV+j4J/fob36LMLm/bRXLQBfTv8HLwPCt38dwVV/Idc+gK9lD/G8YbROv4bw5A8QKzsWjwdyS3J71obKamirb+v/4AeBcDhCdXmIy+eM6VYR6Yo5Y7l2/jiuOG4Ea/c287c3d/Loyt08uXovlUW5nD95OOdPGUF1Wf4ARC8DIRKLs3h9HQ+v2M3Lm/YRd2DWmGI+NncsZx1bQSjXR1FJz9pQVXmIA/XNAxC9iGQCz1Ac7xqJxJz6+pZ0hyEDoKQkH/1fJ6ccJaf8pKYcJaf8vCMQ8FFcGMDzyOdh+d1dbzjrapyFt9LQGCHS6cq2f+8Kcu0DBNc8iLe1lnhuCW0TLyZs3kd0xGwy9TZAmdyGvE07yF33CLlrHyKw5w0cj4/26rMJT/4A7cecCb7AIdv3tg1lukDAR6gwyE0PruDeZVu73O6KOWP53qXTaG4MH5KfcCTGonV1PLZyN69s3k/cgakjC7lgynDONcMpyQ90+ZxDXaYeZ7G4w/JtDTxl9/DcmloawlGGF+Rw4dQRLJw6kmNK8w7ZvrdtKJtlahsS6amKisJXgTlHWqcCkgxqOpGnphwlp/ykphwlp/wcKhDwUVzgh30b8Lx8G6x+xL1TVm4RTFqIM+8GKKuhoSna9ZeSeJScrS+Qax8gd8MTeGJtxIqqaBt/AW3jLyQ6fEZGFZMyrQ15WmrJXf8ouWsfJmfnKwBEKqYfLAY6+RVJ9++TNpTBAgEf+QVBNtU1c+fijTy1cheN4SiFQT/nThnJdfNrqCoP0dKU/It/bVMbT67ey6Mrd7N2bzM+r4e5x5SwYOIwzpgwLOOKSZl0nDmOw1s7G3lq9R6eWVNLXXM7eQEvp40v5/wpIzipqhSft+tzZF+1oWyTSW1IpDdUQJIhSyfy1JSj5JSf1JSj5JSfd/N4IDc3QH4ghi8YOrg8Fm6mJeKjra37E7J62hvJWf8YwXUPE9j2TzzxKLGC0Yli0gVERx4PnqF9z49MaEPeA9vI2fQ0uRufJLD9JTxOnGiZcYtGEy4iXlLTo+fryzaUiTry4w34CAXfKfQ0hyPEI7Ee52fd3mYeX7WbZ9fUsr0hjM8Ds8aWcNbEYZwxcRjDQkN/KOlQP86isTivbz/A4g11PL+2lp0H2sjxeThlXDnnmgpOHVdGMODr9vP1dRvKBkO9DYn0FRWQZMjSiTw15Sg55Sc15Sg55Sc5jweKi/NpaGjp9ZcRT7jeLVKsf4ycLf/AE28nlj+C9uoFtFctIDLmVJycwr4JfID0ZX4GlBPHv+fNRNHoKfx1qwCIloynbcJC2iZcRKx8Up+81JDN0QDpy/w4jsOavc08t2Yvz66pZfP+VjzAcaOKOHVcGSdXl3Hs8BCeIdYDcKi2oaa2KC9t3McL6+t4aeN+Gtui5Pg8zK0q5RxTwWnjyynI7f2UtUM1PwNN7/ciLhWQZMjSiTw15Sg55Sc15Sg55Se1/siRp72RnE3PkrvhcQJbX8Db3ojj9ROpPIH2YxbQXnUmsTIzKIe6eTwQDAbI87+7d01r1Ed4kN4y29NSS862FwlsW0zO5kX4WnbjeLxuzqvPpb3mHGIl4/rltXWcJdcf+XEchw11LTy3ppbFG+pYtbsJgPJQDidXl3JyTRknVpVSGByc99zpOM48/nf3sHGisUF5nMUdB7uniX9trmfJ5v0s39ZALO5Qkhfg1HFlnDa+nBOrSsnP6X5Po+7SMZaaciTiUgFJhiydyFNTjpJTflJTjpJTflLr9xzFIgR2LSNny/PkbH7+YG+YWGgkkdHziIw6icjoecSKa9JeUDp0fp9fwOpHO83vc2Fifp9xg2J+H097E/5dy8jZupicrYvx160EIJ5bTGTMqbTVnEN71Vk4wdJ+j0XHWXIDkZ/a5naWbNrHSxv3s2ST2xvG54HJIwuZPaaE2WOLmTm6iFBO+gtKnef4uWPxBp5eufvgHD/nTBnBdfPHUT0I5vhxHIdt9WGWbq1n6eb9LN1ST0M4CsC48vyDRaNplUVJ5zTqCzrGUlOORFwqIMmQpRN5aspRcspPaspRcspPagOdI2/TDnK2LCKw9UUCO5bga9kDQCw0wi0mVc4lOmIW0fJJ4Bu4uV3eucPY55Lfpj5NdxjzNm4nsHMpgV1L8e9chr9uFR4njuPNIVI5h8iY+bSPnU+04jjw9n0PiGR0nCU30PmJxh3e3nmAlzbu49WtDby9q5Fo3MHngUkjCpk9pphpo4qYNrKQ4YW5AxYXvHOXsa8++FbSW9Wn4y5jkVic1bubeGPHAd7Y3sCbOw6wryUCwPCCHE6oKuXEqhJOGFvCsIKBzZuOsdSUIxFXsgJS+i8hiIiIiPRAvGAU4SkfIjzlQ+A4+Oo3ENj+MoEdLxPYvoTg2ocAcHy5RCumERkxi+jwmUSHTSVWUgPevv/44/FAcYE/dfEIYPndeIDiC2+hrj7W98NsHAdvy278e1ckft7Cv/dNfE073dX+fCIjZtFy/A1EKucSqZwLgbwUTyrZxO/1MGN0MTNGFwMQjsR4c8cBXt3WwGtb6/nza9uJJoo3wwtymDKykGmVRUweUcDEihCl+f1TuPV4IL8gdfEIOHgL++9eMo0D9c19fpxFYnE21LVgdzdh9zSxeo/72BaNAzCqOMiJVaXMGF3E7DElVJflDbm5pUREDterT1DGmMuBbwKTgbnW2mVdbPce4GeAD7jTWntzYnkNcA9QDrwKXG2tbe9NTCIiIpJFPB5ipeOJlY4nPO0qt3jSuJ3A7uX497xOYPdy8t7+Pzxv3AkkikqlE4iVTyJaNolo+SRiJeOIF47uVWEpGAxA3frUxaMOy++GeZ8hN6+acDhydC/qOHha6/DXr8O33/3x71+Lf+9KvK173U3wECsZR6RyLq0jZhOpPIHosCn9UkSTzBUM+JhbVcrcKnc4Y1s0ztq9TazY2ciKnQdYuauRRevqDm5flh9gwrAQEypCjB8WYnx5PmNK8ijOC3T1Et2LIxhgU21zyuJRh3uXbeXa+TVU5geO+jiLxuJsqw+zaV/LwZ8NdS2sq20mEnOrUvkBH8cOD/G+6ZXMGF3EjFFFA97DSERkIPT208MK4H3A7V1tYIzxAb8EzgG2AUuNMQ9ba1cCPwRutdbeY4z5DXAN8OtexiQiIiLZyuMhXjSGtqIxtE28yF0Wi+DbtwZ/3Sr3Z99qAltfJGgfOLib4/UTKxxLrLiaeHEVsaIqYgWVxEMj3J/84eAPdvmyef4YniW39SzUl28j/7wfEg53sYETxxOux9uyG1/jdryN2/A1bsPbuB1f4zZ89RvxttW/s7k/SLRkAu1VZxIdNpVIxXHEhk3BySnoUVwiqeT6vUyrLGJaZREwGoD61gh2dxPraptZV9vM+tpmHnhj58EeOQBFQT9jSvIYUxxkTGkelYW5VBTkMqwgh4qCHEryAniT9NLx+H3c8aLtUax3Lt7INxZOhi4KSG3ROPtb2tnd2MbOA23sPBBmV+JxR0OYbQ1hYvF3ui9VFORQU5bPB2eNZtKIAszwAsaW5iWNW0QkU/SqgGStXQVgjEm22VxgnbV2Q2Lbe4CLjTGrgAXAhxLb/R63N5MKSCIiItJ3fAFiFVOJVUylrdNiT3g//n0WX/0mfA2b8Da4j4GdS/FGmt71NPHcEuL5FTi5xcRzi3Fyi3Fyi3CCxfiKy2DFA+/aJ6m37sdXWElBw36INONpb8TbWnfwxxPej8c5dO4Wx5tDrHAU8cIxtI2/kFjZRLdHVckE4oWjwOM9igSJ9F5JXoATq0s5sfqdSddjcYet9a1s3tfKtvpWtta7j2/vauSZNXuJHzaszO/1UB5yC0mFuT4KgwGKcv0UBt2fsqI8HnljR4/ieviN7ZQX5FDX0Epze4wDrRHqWyPsa3Efm9vfPT9ScdBPZVGQmvJ8zpg4jJqyfKrL8qgqy6cgV733RCR7DcQZcDSwtdPv24ATcYet1Vtro52Wj+7OE/p8HkpK8vs0SBmcfD6v/q9TUI6SU35SU46SU35SG7o5yoeR7/7oEXMcYq37oWkXnsadicdd0LgTb8teCDfgb90L9WuhrQHCB4CjmGAl2gov/IhgTgEEQpBbgJNfDsMm4IROwskfBqFhOKHhUDwWp3gshCrA48WD+yHOD2TCQJmh24YGxlDOT3lZiJnj3r08Eouzt7GN3Y1t7DkQPvi4p7GN/S0RGsMRttaHORCO0NAaOaQnU0+EI3HuXLyB/Bw/eQEfJXkBygpyqBpWQHlBDuWhHMpCOYwsCjKqJI9RxUFCGVgkGsptaKAoRyKppTw7GmOeAUYeYdVN1tqH+j6k1GIxRzPkZwndDSE15Sg55Sc15Sg55Se1zMxREHKqobzaveSVhMeJMawkALdMhnBDD16iBP5zI3X7ujnBbxRo6Gq829CWmW2o72RqfvKBmsIcagpTT7odicUpLMnnlB8+x4HWaMrtOxTn+XnjG+dRW9vYreMs0tpOfWvmTcmaqW2oLylHIq6KisIu16UsIFlrz+7l628Hxnb6fUxiWR1QYozxJ3ohdSwXERERGTIcj49YHHyTL+r+JNoAky8i1t7a93dhE8lAAZ8XL3De1JHdnkQb4LyplTSHIzrORET6wEAMlF8KTDTG1BhjcoAPAg9bax3geeCyxHYfBdLSo0lERESkN1qjPpx5N/RoH2feDbREfP0UkUjmcaIxrpt/hPFwSVw3v4Z45N3zHImISM/1qoBkjLnUGLMNmAc8aox5MrF8lDHmMYBE76LPAE8Cq4B7rbVvJ57iS8D/M8asw+0gfldv4hERERFJh3A4AmXjYNZV3dth1tVQVkNb29HdWlwkG4XDEarLQ1w+Z0y3tr9izliqykM6zkRE+khv78L2IPDgEZbvAC7o9PtjwGNH2G4D7l3aRERERIYsx4GGpijFC3+KBw8sv7vrjWddjbPwVhoaNaxGpCccB1qawnz/0uPw4OHeZVu73PaKOWP53qXTaG4M6zgTEekjmXeLAREREZE0iERiNDRC8YW3wLzP4Hn5Nlj9CLQdgNwimLTQHeZWVkNDY4SIhtWI9FgkEqO5Mcx3L5nGtfNruHPxRp5auYvGcJTCoJ9zp4zkuvk1VJWHaG4M6zgTEelDKiCJiIiI9JFIJEZdfYzcvGryz/shvotvO7guFm6mJeKjrb5NPSJEeiESiXGgvpnK/ADfWDiZH102/eC65nCEeGK9jjMRkb6lApKIiIhIH3Icd66WcBg8TY0UF+fT0NCS+DIbT3d4Ihmh4zgjHKG1KXzYcSYiIv1hIO7CJiIiIpKVOr7M6kutSP/RcSYiMjBUQBIRERERERERkaRUQBIRERERERERkaRUQBIRERERERERkaQ8ztAcLLwX2JzuIEREREREREREMkgVUHGkFUO1gCQiIiIiIiIiIgNEQ9hERERERERERCQpFZBERERERERERCQpFZBERERERERERCQpFZBERERERERERCQpFZBERERERERERCQpf7oDEAEwxjRZawvSHcdgZIyJAW91WnSJtXZTF9suAm601i4bgNAGBWOMA/zRWntV4nc/sBN4xVq7MK3BDSLGmEuAB4HJ1trVaQ5n0FD76T6dp7svVa6y9Fx9CToHHZEx5ibgQ0AMiAOfsta+0sPnOANot9a+1PcRDh7GmDHAL4EpuBfCHwG+aK1t72L7zwG/tda2DFiQaZJ4P7vFWvuFxO83AgXW2m+mNbBBotPn6QAQBf4A3Gqtjac1MJEhRj2QRAa/VmvtzE4/m9Id0CDTDEwzxuQlfj8H2N6TJ0gUDTLdlcCLicduM8b4+iecQaPX7UdEuuWozkGZzhgzD1gIzLbWTgfOBrYexVOdAZzch6ENOsYYD/BX4G/W2onAsUAB8L0ku30OyO//6AaFNuB9xphh6Q5kkOr4PD0V973+fOAbaY5JZMjJhi9NMkQYYwqAh4BS3KsD/2WtfcgYUw08jvvB82TcL3cXW2tb0xVruhljjgduwf3gVAt8zFq7M7H6amPMnbjH9yestf9KU5gD6THgQuB+3C8nfwbmAxhj5gI/A4JAK/Bxa601xnwMeB9uDn3A6QMf9sBIHFunAmcCfwe+kbha/W2gEZgAPA9cb62NG2OagNtxv8j8O+6xl8mOpv28APyHtfb1xHYvAv9urX1j4MMfOIl2c2NH7yxjzG3AMmvt/xpjNgG/By7CPYdfns09TZLlKp1xpUOSc1BXbekC3Pe4ZuCfwLgM7hFYCdRaa9sArLW10PX7fKL32hu471l+4BPAHuDTQMwYcxVwg7V28UD/IQNgARC21v4OwFobM8Z8HthojPkm8C3gPbi9uO4APMAo4HljTK219sz0hD1gosBvgc8DN3Vekfgs/T/AMGAv8HGgAXgTqEm894eA1bjHW2QA4x5w1to9xphPAksTbccL3IxbiM0FfmmtvR3AGPMl4CrcdvW4tfbLaQlaZJBQDyQZTMLApdba2bgfMn+SuNoEMBH3ZD4VqAfen54Q0yLPGPN64udBY0wA+AVwmbX2eNwPBJ2vvuVba2cC1yfWZYN7gA8aY4LAdKBz1//VwHxr7Szg68D3O62bjZvHjC0eJVwMPGGtXQPUJb6YAMwFbsAdCjAet6AGEMIdwjXDWpvpxSM4uvZzF/AxAGPMsUAw04tH3VSbOIf/Grgx3cHIoNHVOehdEsfh7cD5ife4igGKMV2eAsYaY9YYY35ljDm9p+/ziZ7Jv8EdjjMzQ4tHAFOBVzsvsNYeALYA1wLVwMxET64/Wmt/DuwAzsyC4lGHXwIfNsYUH7b8F8DvO3ID/Nxa2wC8zjsX0BYCT2Z68aiDtXYD7gXE4cA1QIO19gTgBOA6Y0yNMeZ83PPXidbaGcCP0hawyCChApIMJh7g+8aYN4FngNHAiMS6jR1X+nE/PFQPeHTp03kI26WAAaYBTxtjXgf+CxjTafs/A1hrXwCKjDElAxzvgLPWvonbJq7E7U3SWTFwnzFmBXAr7gfQDk9ba/cNSJDpdSVukYTEY8cQkn9ZazdYa2O47ebUxPIY8MDAhpg+R9l+7gMWJr7ofQL43wEJdvD7a+Ix287TklxX56AjmQRssNZuTPz+5/4MLN2stU3A8cAncXuG/AX4FHqf76kzgNuttVGALHlvf5dEQe0PwH8ctmoe8KfEv+/mnff7vwAfSPz7g4nfs9G5wEcSx9srQDnuxeuzgd91zKGVre1KpDMNYZPB5MO4VxqPt9ZGEsMhgol1bZ22iwF5ZC8P8La1dl4X650Uv2eqh4Ef436ILO+0/DvA89baSxNduBd1Wtc8UMGlizGmDLfb/3GJCTZ9uG3iUbpuK+FEUSmb9Kj9WGtbjDFP416ZvAL3C2A2iHLoxafgYes7ztUx9BkjVa6yQpJz0EMoP4A7FAv33LLIGPMW7tBhvc+/20rgss4LjDFFwDHApnQENEj9FHgN+F03tn0Y9+JtGe772HP9GNegYowZh/tetQf3s/UN1tonD9vmvHTEJjKYqQeSDCbFwJ5E8ehMoCrdAQ1SFqhITLyJMSZgjOncq+YDieWn4nbHbUhDjOnwP8C3rLVvHba8mHcmRf7YgEY0OFwG3G2trbLWVltrxwIbcef4mZvoou3FbTfZMFytK0fTfu4Efg4stdbu79/wBo3NwBRjTG6i18NZaY5nMFOuXF2dg7wcOT8WGJco2MI7vSMyknFN7LRoJrCKnr/PNwKFAxN12jwL5BtjPgIHb/LwE9weoE8Cn+q4KUaiIALZkZdDJHrJ3Is7LKvDS7g9jMC9YLs4sW0TsBR3rr9HsuXikTGmAnfY523WWge3/fxbolcxxphjE3NCPQ183BiTn1he1tVzimQLFZAk7RJv9m24Y7LnJK6+fQR37hE5TOJWtZcBPzTGvIE7fr3znVfCxpjluG+M17z7GTKTtXZbYr6Dw/0I+EEiJ9nYI+JK3Ftnd/ZAYvlS4DbcLysbj7Bd1jia9mOtfRU4QPeu8g5pHedpa+1W3C8mKxKPy9Ma2CCkXL1LV+egD3KE/CRukHE98IQx5lXcAkAmXwgpAH5vjFmZGMI/BXe+tZ6+z/8duDQxX+L8AYt+ACW+6F8KXG6MWQuswZ0/86u4Bf0twJuJnH0osdtvcdvS82kIOZ1+gjthdocbcAshbwJXA5/ttO4vuJNEZ/rwtY45Rd/GnSrjKdyJ18FtPyuB1xJD1m8H/NbaJ3B7aS1LDG/T3H6S9TyOky29XmWwMsbMAO6w1s5Ndywi2eLwOyBJzxljRuEOO5lkrY2nOZx+pfN09ylXvWeMKbDWNiVupPFLYK219tZ0xzUYJO7CdqO1dlm6YxERkeyjHkiSVsaYT+NOBvlf6Y5FRKS7EkMoXgFuyoLikc7T3aRc9ZnrElf738YdRnp7esMRERERUA8kERERERERERFJQT2QREREREREREQkKRWQREREREREREQkKRWQREREREREREQkKRWQREREREREREQkKRWQREREREREREQkKRWQREREREREREQkqf8PsAPrI2HA5ZcAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "months = range(0, 12)\n", + "month_labels = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec']\n", + "# sin月份特征\n", + "months_sin = map(lambda x: math.sin(2 * math.pi * x / len(months)), months)\n", + "# cos月份特征\n", + "months_cos = map(lambda x: math.cos(2 * math.pi * x / len(months)), months)\n", + "\n", + "# 绘制每个月的月份特征组合\n", + "plt.figure(figsize=(20, 5))\n", + "x_axis = np.arange(-1, 13, 1e-2)\n", + "sns.lineplot(x=x_axis, y=np.sin(2 * math.pi * x_axis / len(months)))\n", + "sns.lineplot(x=x_axis, y=np.cos(2 * math.pi * x_axis / len(months)))\n", + "sns.scatterplot(x=months, y=months_sin, s=200)\n", + "sns.scatterplot(x=months, y=months_cos, s=200)\n", + "plt.xticks(ticks=months, labels=month_labels)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.042371, + "end_time": "2021-11-10T09:15:05.235016", + "exception": false, + "start_time": "2021-11-10T09:15:05.192645", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "构造SODA数据的sin月份特征。" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:05.382538Z", + "iopub.status.busy": "2021-11-10T09:15:05.362189Z", + "iopub.status.idle": "2021-11-10T09:15:09.991637Z", + "shell.execute_reply": "2021-11-10T09:15:09.991174Z", + "shell.execute_reply.started": "2021-10-21T07:25:02.899195Z" + }, + "papermill": { + "duration": 4.714823, + "end_time": "2021-11-10T09:15:09.991777", + "exception": false, + "start_time": "2021-11-10T09:15:05.276954", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(100, 36, 24, 72)" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造一个维度为100*36*24*72的矩阵，矩阵中的每个值为所在月份的sin函数值\n", + "soda_month_sin = np.zeros((100, 36, 24, 72))\n", + "for y in range(100):\n", + " for m in range(36):\n", + " for lat in range(24):\n", + " for lon in range(72):\n", + " soda_month_sin[y, m, lat, lon] = math.sin(2 * math.pi * (m % 12) / 12)\n", + " \n", + "soda_month_sin.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.042795, + "end_time": "2021-11-10T09:15:10.077774", + "exception": false, + "start_time": "2021-11-10T09:15:10.034979", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "构造SODA数据的cos月份特征。" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:10.213213Z", + "iopub.status.busy": "2021-11-10T09:15:10.203033Z", + "iopub.status.idle": "2021-11-10T09:15:15.014679Z", + "shell.execute_reply": "2021-11-10T09:15:15.015236Z", + "shell.execute_reply.started": "2021-10-21T07:25:07.740353Z" + }, + "papermill": { + "duration": 4.894975, + "end_time": "2021-11-10T09:15:15.015420", + "exception": false, + "start_time": "2021-11-10T09:15:10.120445", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(100, 36, 24, 72)" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造一个维度为100*36*24*72的矩阵，矩阵中的每个值为所在月份的cos函数值\n", + "soda_month_cos = np.zeros((100, 36, 24, 72))\n", + "for y in range(100):\n", + " for m in range(36):\n", + " for lat in range(24):\n", + " for lon in range(72):\n", + " soda_month_cos[y, m, lat, lon] = math.cos(2 * math.pi * (m % 12) / 12)\n", + " \n", + "soda_month_cos.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.043675, + "end_time": "2021-11-10T09:15:15.104012", + "exception": false, + "start_time": "2021-11-10T09:15:15.060337", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "构造CMIP数据的sin月份特征。" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:15:15.231788Z", + "iopub.status.busy": "2021-11-10T09:15:15.216440Z", + "iopub.status.idle": "2021-11-10T09:19:00.373213Z", + "shell.execute_reply": "2021-11-10T09:19:00.373662Z", + "shell.execute_reply.started": "2021-10-21T07:25:12.923041Z" + }, + "papermill": { + "duration": 225.2277, + "end_time": "2021-11-10T09:19:00.373825", + "exception": false, + "start_time": "2021-11-10T09:15:15.146125", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(4645, 36, 24, 72)" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造一个维度为4645*36*24*72的矩阵，矩阵中的每个值为所在月份的sin函数值\n", + "cmip_month_sin = np.zeros((4645, 36, 24, 72))\n", + "for y in range(4645):\n", + " for m in range(36):\n", + " for lat in range(24):\n", + " for lon in range(72):\n", + " cmip_month_sin[y, m, lat, lon] = math.sin(2 * math.pi * (m % 12) / 12)\n", + " \n", + "cmip_month_sin.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.043105, + "end_time": "2021-11-10T09:19:00.460782", + "exception": false, + "start_time": "2021-11-10T09:19:00.417677", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "构造CMIP数据的cos月份特征。" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:19:00.580015Z", + "iopub.status.busy": "2021-11-10T09:19:00.564652Z", + "iopub.status.idle": "2021-11-10T09:22:46.383555Z", + "shell.execute_reply": "2021-11-10T09:22:46.384007Z", + "shell.execute_reply.started": "2021-10-21T07:29:05.42979Z" + }, + "papermill": { + "duration": 225.880432, + "end_time": "2021-11-10T09:22:46.384183", + "exception": false, + "start_time": "2021-11-10T09:19:00.503751", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(4645, 36, 24, 72)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造一个维度为4645*36*24*72的矩阵，矩阵中的每个值为所在月份的cos函数值\n", + "cmip_month_cos = np.zeros((4645, 36, 24, 72))\n", + "for y in range(4645):\n", + " for m in range(36):\n", + " for lat in range(24):\n", + " for lon in range(72):\n", + " cmip_month_cos[y, m, lat, lon] = math.cos(2 * math.pi * (m % 12) / 12)\n", + " \n", + "cmip_month_cos.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.042597, + "end_time": "2021-11-10T09:22:46.469951", + "exception": false, + "start_time": "2021-11-10T09:22:46.427354", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 数据扁平化" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.042975, + "end_time": "2021-11-10T09:22:46.555947", + "exception": false, + "start_time": "2021-11-10T09:22:46.512972", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "在Task2中我们发现，赛题中给出的数据量非常少，如何增加数据量呢？对于时序数据，一种常用的做法就是滑窗。\n", + "\n", + "由于每条数据在时间上有重叠，我们取数据的前12个月拼接起来，就得到了长度为（数据条数×12个月）的序列数据，如图1所示：\n", + "\n", + "\n", + "然后我们以每个月为起始月，接下来的12个月作为模型输入X，后24个月的Nino3.4指数作为预测目标Y构建训练样本，如图2所示：\n", + "\n", + "\n", + "需要注意的是，CMIP数据提供了不同的拟合模式，只有在同种模式下各个年份的数据在时间上是连续的，因此同种模式的数据才能在时间上拼接起来，除去最后11个月不能构成训练样本外，滑窗最终能获得的训练样本数量可以按以下方式计算得到：\n", + "\n", + "- SODA：1种模式×（100年×12-11）=1189条样本\n", + "- CMIP6：15种模式×（151年×12-11）=27015条样本\n", + "- CMIP5：17种模式×（140年×12-11）=28373条样本" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.042529, + "end_time": "2021-11-10T09:22:46.641383", + "exception": false, + "start_time": "2021-11-10T09:22:46.598854", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "在下面的代码中，我们只将各个模式的数据拼接起来而没有采用滑窗，这是因为考虑到采用滑窗得到的训练样本维度是（数据条数×12×24×72），需要占用大量的内存资源。我们在之后构建数据集时，随机抽取了部分样本，大家在实际问题中，如果资源足够的话，可以采用滑窗构建的全部的数据，不过需要注意数据量大的情况下可以考虑构建更深的模型来挖掘更多信息。" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:22:46.740678Z", + "iopub.status.busy": "2021-11-10T09:22:46.737178Z", + "iopub.status.idle": "2021-11-10T09:22:46.742965Z", + "shell.execute_reply": "2021-11-10T09:22:46.742544Z", + "shell.execute_reply.started": "2021-10-21T07:32:53.823861Z" + }, + "papermill": { + "duration": 0.057956, + "end_time": "2021-11-10T09:22:46.743082", + "exception": false, + "start_time": "2021-11-10T09:22:46.685126", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 数据扁平化\n", + "def make_flatted(train_ds, label_ds, month_sin, month_cos, info, start_idx=0):\n", + " keys = ['sst', 't300', 'ua', 'va']\n", + " label_key = 'nino'\n", + " # 年数\n", + " years = info[1]\n", + " # 模式数\n", + " models = info[2]\n", + " \n", + " train_list = []\n", + " label_list = []\n", + " \n", + " # 将同种模式下的数据拼接起来\n", + " for model_i in range(models):\n", + " blocks = []\n", + " \n", + " # 对每个特征，取每条数据的前12个月进行拼接\n", + " for key in keys:\n", + " block = train_ds[key][start_idx + model_i * years: start_idx + (model_i + 1) * years, :12].reshape(-1, 24, 72, 1).data\n", + " blocks.append(block)\n", + " # 增加sin月份特征\n", + " block_sin = month_sin[start_idx + model_i * years: start_idx + (model_i + 1) * years, :12].reshape(-1, 24, 72, 1)\n", + " blocks.append(block_sin)\n", + " # 增加cos月份特征\n", + " block_cos = month_cos[start_idx + model_i * years: start_idx + (model_i + 1) * years, :12].reshape(-1, 24, 72, 1)\n", + " blocks.append(block_cos)\n", + " \n", + " # 将所有特征在最后一个维度上拼接起来\n", + " train_flatted = np.concatenate(blocks, axis=-1)\n", + " \n", + " # 取12-23月的标签进行拼接，注意加上最后一年的最后12个月的标签（与最后一年12-23月的标签共同构成最后一年前12个月的预测目标）\n", + " label_flatted = np.concatenate([\n", + " label_ds[label_key][start_idx + model_i * years: start_idx + (model_i + 1) * years, 12: 24].reshape(-1).data,\n", + " label_ds[label_key][start_idx + (model_i + 1) * years - 1, 24: 36].reshape(-1).data\n", + " ], axis=0)\n", + " \n", + " train_list.append(train_flatted)\n", + " label_list.append(label_flatted)\n", + " \n", + " return train_list, label_list" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:22:46.835870Z", + "iopub.status.busy": "2021-11-10T09:22:46.835071Z", + "iopub.status.idle": "2021-11-10T09:24:24.878233Z", + "shell.execute_reply": "2021-11-10T09:24:24.878703Z", + "shell.execute_reply.started": "2021-10-21T07:32:53.840256Z" + }, + "papermill": { + "duration": 98.090322, + "end_time": "2021-11-10T09:24:24.878897", + "exception": false, + "start_time": "2021-11-10T09:22:46.788575", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((1, 1200, 24, 72, 6), (15, 1812, 24, 72, 6), (17, 1680, 24, 72, 6))" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "soda_info = ('soda', 100, 1)\n", + "cmip6_info = ('cmip6', 151, 15)\n", + "cmip5_info = ('cmip5', 140, 17)\n", + "\n", + "soda_trains, soda_labels = make_flatted(soda_train, soda_label, soda_month_sin, soda_month_cos, soda_info)\n", + "cmip6_trains, cmip6_labels = make_flatted(cmip_train, cmip_label, cmip_month_sin, cmip_month_cos, cmip6_info)\n", + "cmip5_trains, cmip5_labels = make_flatted(cmip_train, cmip_label, cmip_month_sin, cmip_month_cos, cmip5_info, cmip6_info[1]*cmip6_info[2])\n", + "\n", + "# 得到扁平化后的数据维度为（模式数×序列长度×纬度×经度×特征数），其中序列长度=年数×12\n", + "np.shape(soda_trains), np.shape(cmip6_trains), np.shape(cmip5_trains)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:25.152679Z", + "iopub.status.busy": "2021-11-10T09:24:24.982782Z", + "iopub.status.idle": "2021-11-10T09:24:25.154686Z", + "shell.execute_reply": "2021-11-10T09:24:25.155159Z", + "shell.execute_reply.started": "2021-10-21T07:33:48.040591Z" + }, + "papermill": { + "duration": 0.229381, + "end_time": "2021-11-10T09:24:25.155316", + "exception": false, + "start_time": "2021-11-10T09:24:24.925935", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "del soda_month_sin, soda_month_cos\n", + "del cmip_month_sin, cmip_month_cos\n", + "del soda_train, soda_label\n", + "del cmip_train, cmip_label" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.04346, + "end_time": "2021-11-10T09:24:25.242948", + "exception": false, + "start_time": "2021-11-10T09:24:25.199488", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 空值填充" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.04333, + "end_time": "2021-11-10T09:24:25.330153", + "exception": false, + "start_time": "2021-11-10T09:24:25.286823", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "在Task2中我们发现，除SST外，其它特征中都存在空值，这些空值基本都在陆地上，因此我们直接将空值填充为0。" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:25.422928Z", + "iopub.status.busy": "2021-11-10T09:24:25.421803Z", + "iopub.status.idle": "2021-11-10T09:24:25.494885Z", + "shell.execute_reply": "2021-11-10T09:24:25.495301Z", + "shell.execute_reply.started": "2021-10-21T07:33:48.224046Z" + }, + "papermill": { + "duration": 0.121727, + "end_time": "2021-11-10T09:24:25.495442", + "exception": false, + "start_time": "2021-11-10T09:24:25.373715", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of null in soda_trains after fillna: 0\n" + ] + } + ], + "source": [ + "# 填充SODA数据中的空值\n", + "soda_trains = np.array(soda_trains)\n", + "soda_trains_nan = np.isnan(soda_trains)\n", + "soda_trains[soda_trains_nan] = 0\n", + "print('Number of null in soda_trains after fillna:', np.sum(np.isnan(soda_trains)))" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:25.587773Z", + "iopub.status.busy": "2021-11-10T09:24:25.586691Z", + "iopub.status.idle": "2021-11-10T09:24:27.292695Z", + "shell.execute_reply": "2021-11-10T09:24:27.291961Z", + "shell.execute_reply.started": "2021-10-21T07:33:48.30097Z" + }, + "papermill": { + "duration": 1.753544, + "end_time": "2021-11-10T09:24:27.292852", + "exception": false, + "start_time": "2021-11-10T09:24:25.539308", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of null in cmip6_trains after fillna: 0\n" + ] + } + ], + "source": [ + "# 填充CMIP6数据中的空值\n", + "cmip6_trains = np.array(cmip6_trains)\n", + "cmip6_trains_nan = np.isnan(cmip6_trains)\n", + "cmip6_trains[cmip6_trains_nan] = 0\n", + "print('Number of null in cmip6_trains after fillna:', np.sum(np.isnan(cmip6_trains)))" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:27.385709Z", + "iopub.status.busy": "2021-11-10T09:24:27.384660Z", + "iopub.status.idle": "2021-11-10T09:24:29.161437Z", + "shell.execute_reply": "2021-11-10T09:24:29.160639Z", + "shell.execute_reply.started": "2021-10-21T07:33:50.049562Z" + }, + "papermill": { + "duration": 1.824903, + "end_time": "2021-11-10T09:24:29.161584", + "exception": false, + "start_time": "2021-11-10T09:24:27.336681", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Number of null in cmip6_trains after fillna: 0\n" + ] + } + ], + "source": [ + "# 填充CMIP5数据中的空值\n", + "cmip5_trains = np.array(cmip5_trains)\n", + "cmip5_trains_nan = np.isnan(cmip5_trains)\n", + "cmip5_trains[cmip5_trains_nan] = 0\n", + "print('Number of null in cmip6_trains after fillna:', np.sum(np.isnan(cmip5_trains)))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.044652, + "end_time": "2021-11-10T09:24:29.250661", + "exception": false, + "start_time": "2021-11-10T09:24:29.206009", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 构造数据集" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.043586, + "end_time": "2021-11-10T09:24:29.337940", + "exception": false, + "start_time": "2021-11-10T09:24:29.294354", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "在划分训练/验证集时，一个需要考虑的问题是训练集、验证集、测试集三者的分布是否是一致的。在本赛题中我们拿到的是两份数据，其中CMIP数据是CMIP5/6模式模拟的历史数据，SODA数据是由SODA模式重建的的历史观测同化数据，线上测试集则是来自国际多个海洋资料的同化数据，由此看来，SODA数据和线上测试集的分布是较为一致的，CMIP数据的分布则与测试集不同。在三者不一致的情况下，我们通常会尽可能使验证集与测试集的分布一致，这样当模型在验证集上有较好的表现时，在测试集上也会有较好的表现。\n", + "\n", + "因此，我们从CMIP数据的每个模式中各抽取100条数据作为训练集（这里抽取的样本数只是作为一个示例，实际模型训练的时候使用多少样本需要综合考虑可用的资源条件和构建的模型深度），从SODA模式中抽取100条数据作为验证集。有的同学可能会疑惑，既然这里只用了100条SODA数据，那么为什么还要对SODA数据扁平化后再抽样而不直接用原始数据呢，因为直接取原始数据的前12个月作为输入，后24个月作为标签所得到的验证集每一条都是从0月开始的，而线上的测试集起始月份是随机抽取的，因此这里仍然要尽可能保证验证集与测试集的数据分布一致，使构建的验证集的起始月份也是随机的。\n", + "\n", + "我们这里没有构造测试集，因为线上的测试集已经公开了，可以直接使用，在比赛时，线上的测试集是保密的，需要构造线下的测试集来评估模型效果，同时需要注意线下的评估结果和线上的提交结果是否差距不大或者变化趋势是一致的，如果不是就需要调整线下的测试集，保证它和线上测试集的分布尽可能一致，能够较为准确地指示模型的调整方向。" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:29.445733Z", + "iopub.status.busy": "2021-11-10T09:24:29.444632Z", + "iopub.status.idle": "2021-11-10T09:24:30.366281Z", + "shell.execute_reply": "2021-11-10T09:24:30.365467Z", + "shell.execute_reply.started": "2021-10-21T07:33:51.83364Z" + }, + "papermill": { + "duration": 0.984392, + "end_time": "2021-11-10T09:24:30.366438", + "exception": false, + "start_time": "2021-11-10T09:24:29.382046", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 构造训练集\n", + "\n", + "X_train = []\n", + "y_train = []\n", + "# 从CMIP5的17种模式中各抽取100条数据\n", + "for model_i in range(17):\n", + " samples = np.random.choice(cmip5_trains.shape[1]-12, size=100)\n", + " for ind in samples:\n", + " X_train.append(cmip5_trains[model_i, ind: ind+12])\n", + " y_train.append(cmip5_labels[model_i][ind: ind+24])\n", + "# 从CMIP6的15种模式种各抽取100条数据\n", + "for model_i in range(15):\n", + " samples = np.random.choice(cmip6_trains.shape[1]-12, size=100)\n", + " for ind in samples:\n", + " X_train.append(cmip6_trains[model_i, ind: ind+12])\n", + " y_train.append(cmip6_labels[model_i][ind: ind+24])\n", + "X_train = np.array(X_train)\n", + "y_train = np.array(y_train)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:30.464643Z", + "iopub.status.busy": "2021-11-10T09:24:30.463377Z", + "iopub.status.idle": "2021-11-10T09:24:30.496402Z", + "shell.execute_reply": "2021-11-10T09:24:30.495893Z", + "shell.execute_reply.started": "2021-10-21T07:33:52.757012Z" + }, + "papermill": { + "duration": 0.083522, + "end_time": "2021-11-10T09:24:30.496543", + "exception": false, + "start_time": "2021-11-10T09:24:30.413021", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 构造测试集\n", + "\n", + "X_valid = []\n", + "y_valid = []\n", + "samples = np.random.choice(soda_trains.shape[1]-12, size=100)\n", + "for ind in samples:\n", + " X_valid.append(soda_trains[0, ind: ind+12])\n", + " y_valid.append(soda_labels[0][ind: ind+24])\n", + "X_valid = np.array(X_valid)\n", + "y_valid = np.array(y_valid)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:30.635941Z", + "iopub.status.busy": "2021-11-10T09:24:30.635228Z", + "iopub.status.idle": "2021-11-10T09:24:30.637802Z", + "shell.execute_reply": "2021-11-10T09:24:30.638226Z", + "shell.execute_reply.started": "2021-10-21T07:33:52.794851Z" + }, + "papermill": { + "duration": 0.05296, + "end_time": "2021-11-10T09:24:30.638353", + "exception": false, + "start_time": "2021-11-10T09:24:30.585393", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((3200, 12, 24, 72, 6), (3200, 24), (100, 12, 24, 72, 6), (100, 24))" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 查看数据集维度\n", + "X_train.shape, y_train.shape, X_valid.shape, y_valid.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:30.736251Z", + "iopub.status.busy": "2021-11-10T09:24:30.735238Z", + "iopub.status.idle": "2021-11-10T09:24:30.743722Z", + "shell.execute_reply": "2021-11-10T09:24:30.744128Z", + "shell.execute_reply.started": "2021-10-21T07:33:52.802917Z" + }, + "papermill": { + "duration": 0.061876, + "end_time": "2021-11-10T09:24:30.744261", + "exception": false, + "start_time": "2021-11-10T09:24:30.682385", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "del cmip5_trains, cmip5_labels\n", + "del cmip6_trains, cmip6_labels\n", + "del soda_trains, soda_labels" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:24:30.837742Z", + "iopub.status.busy": "2021-11-10T09:24:30.835677Z", + "iopub.status.idle": "2021-11-10T09:24:37.325218Z", + "shell.execute_reply": "2021-11-10T09:24:37.325813Z", + "shell.execute_reply.started": "2021-10-21T07:33:52.823277Z" + }, + "papermill": { + "duration": 6.537288, + "end_time": "2021-11-10T09:24:37.326024", + "exception": false, + "start_time": "2021-11-10T09:24:30.788736", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 保存数据集\n", + "np.save('X_train_sample.npy', X_train)\n", + "np.save('y_train_sample.npy', y_train)\n", + "np.save('X_valid_sample.npy', X_valid)\n", + "np.save('y_valid_sample.npy', y_valid)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 18.052826, + "end_time": "2021-11-10T09:24:55.435219", + "exception": false, + "start_time": "2021-11-10T09:24:37.382393", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "### 模型构建" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 1.858781, + "end_time": "2021-11-10T09:24:59.616340", + "exception": false, + "start_time": "2021-11-10T09:24:57.757559", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "在模型构建部分的通用流程是：构造评估函数 -> 构建并训练模型 -> 模型评估，后两步是循环的，可以根据评估结果重新调整并训练模型，再重新进行评估。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.044934, + "end_time": "2021-11-10T09:25:00.686854", + "exception": false, + "start_time": "2021-11-10T09:25:00.641920", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 构造评估函数" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.045047, + "end_time": "2021-11-10T09:25:00.776034", + "exception": false, + "start_time": "2021-11-10T09:25:00.730987", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "模型的评估函数通常就是官方给出的评估指标，不过在比赛中经常会出现线下的评估结果和提交后的线上评估结果不一致的情况，这通常是线下测试集和线上测试集分布不一致造成的。" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:00.870467Z", + "iopub.status.busy": "2021-11-10T09:25:00.869929Z", + "iopub.status.idle": "2021-11-10T09:25:22.612495Z", + "shell.execute_reply": "2021-11-10T09:25:22.611911Z", + "shell.execute_reply.started": "2021-11-10T08:24:24.814274Z" + }, + "papermill": { + "duration": 21.791722, + "end_time": "2021-11-10T09:25:22.612629", + "exception": false, + "start_time": "2021-11-10T09:25:00.820907", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 读取数据集\n", + "X_train = np.load('../input/ai-earth-task03-samples/X_train_sample.npy')\n", + "y_train = np.load('../input/ai-earth-task03-samples/y_train_sample.npy')\n", + "X_valid = np.load('../input/ai-earth-task03-samples/X_valid_sample.npy')\n", + "y_valid = np.load('../input/ai-earth-task03-samples/y_valid_sample.npy')" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:22.708325Z", + "iopub.status.busy": "2021-11-10T09:25:22.705056Z", + "iopub.status.idle": "2021-11-10T09:25:22.710690Z", + "shell.execute_reply": "2021-11-10T09:25:22.711121Z", + "shell.execute_reply.started": "2021-11-10T08:24:50.053353Z" + }, + "papermill": { + "duration": 0.053184, + "end_time": "2021-11-10T09:25:22.711246", + "exception": false, + "start_time": "2021-11-10T09:25:22.658062", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((3200, 12, 24, 72, 6), (3200, 24), (100, 12, 24, 72, 6), (100, 24))" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "X_train.shape, y_train.shape, X_valid.shape, y_valid.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:22.807577Z", + "iopub.status.busy": "2021-11-10T09:25:22.804732Z", + "iopub.status.idle": "2021-11-10T09:25:22.809401Z", + "shell.execute_reply": "2021-11-10T09:25:22.809787Z", + "shell.execute_reply.started": "2021-11-10T08:24:50.070165Z" + }, + "papermill": { + "duration": 0.053678, + "end_time": "2021-11-10T09:25:22.809948", + "exception": false, + "start_time": "2021-11-10T09:25:22.756270", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 构造数据管道\n", + "class AIEarthDataset(Dataset):\n", + " def __init__(self, data, label):\n", + " self.data = torch.tensor(data, dtype=torch.float32)\n", + " self.label = torch.tensor(label, dtype=torch.float32)\n", + "\n", + " def __len__(self):\n", + " return len(self.label)\n", + " \n", + " def __getitem__(self, idx):\n", + " return self.data[idx], self.label[idx]" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:22.919404Z", + "iopub.status.busy": "2021-11-10T09:25:22.918885Z", + "iopub.status.idle": "2021-11-10T09:25:24.047360Z", + "shell.execute_reply": "2021-11-10T09:25:24.048122Z", + "shell.execute_reply.started": "2021-11-10T08:24:50.081715Z" + }, + "papermill": { + "duration": 1.193094, + "end_time": "2021-11-10T09:25:24.048294", + "exception": false, + "start_time": "2021-11-10T09:25:22.855200", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "batch_size = 32\n", + "\n", + "trainset = AIEarthDataset(X_train, y_train)\n", + "trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True)\n", + "\n", + "validset = AIEarthDataset(X_valid, y_valid)\n", + "validloader = DataLoader(validset, batch_size=batch_size, shuffle=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.044471, + "end_time": "2021-11-10T09:25:24.138027", + "exception": false, + "start_time": "2021-11-10T09:25:24.093556", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 模型构造与训练" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.044751, + "end_time": "2021-11-10T09:25:24.227333", + "exception": false, + "start_time": "2021-11-10T09:25:24.182582", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "这部分是赛题的重点，该TOP方案采用的是CNN+LSTM的串行结构，其中CNN用来提取空间信息，LSTM用来提取时间信息，模型结构如下图所示。\n", + "\n", + "\n", + "\n", + "- CNN部分\n", + "\n", + "CNN常用于处理图像信息，它在处理空间信息上也有很好的表现。CNN的输入尺寸是（N,C,H,W），其中N是批量梯度下降中一个批次的样本数量，H和W分别是输入图像的高和宽，C是输入图像的通道数，对于本题中的空间数据，H和W就对应数据的纬度和经度，C对应特征数。我们的训练样本中还多了一个时间维度，因此需要用将输入数据的格式(N,T,H,W,C)转换为(N×T,C,H,W)。\n", + "\n", + "BatchNormalization（后面简称BN）是批标准化层，通常放在卷积层后用于标准化数据的分布，能够减少各层不同数据分布之间的相互影响和依赖，具有加快模型训练速度、避免梯度爆炸、在一定程度上能增强模型泛化能力等优点，是神经网络问题中常用的“大杀器”。不过目前关于BN层和ReLU激活函数的放置顺序孰先孰后的问题众说纷纭，具体还是看模型的效果。关于这个问题的讨论可以参考https://www.zhihu.com/question/283715823\n", + "\n", + "总体来看CNN这一部分采用的是比较通用的结构，第一层采用比较大的卷积核（7×7），后面接多层的小卷积核（3×3），并用BN提升模型效果，用池化层减少模型参数、扩大感受野，池化层常用的有MaxPooling和AveragePooling，通常MaxPooling效果更好，不过具体看模型效果。模型的主要难点就在于调参，目前模型调参没有标准的答案，更多地是参考前人的经验以及不断地尝试。\n", + "\n", + "- LSTM部分\n", + "\n", + "CNN部分经过Flatten层将除时间维度以外的维度压平（即除时间步长12外的其它维度大小相乘，例如CNN部分最后的池化层输出维度是（N,T,C,H,W），则压平后的维度是（N,T,C×H×W）），输入LSTM层。LSTM层接受的输入维度为（Time_steps，Input_size），其中Time_steps就是时间步长12，Input_size是压平后的维度大小。Pytorch中LSTM的主要参数是input_size、hidden_size（隐层节点数）、batch_first（一个批次的样本数量N是否在第1维度），batch_first为True时输入和输出的数据格式为(N,T,input_size/hidden_size)，为数据格式为(T,N,input_size/hidden_size)，需要注意的一点是LSTM的输出形式是tensor格式的output和tuple格式的(h_n,c_n)，其中output是所有时间步的输出(N,T,hidden_size)，h_n是隐层的输出（即最后一个时间步的输出，格式为(1,N,hidden_size)），c_n是记忆细胞cell的输出。因为我们通过多层LSTM要获得的并非一个时间序列，而是要抽取出一个关于输入序列的特征表达，因此最后我们使用最后一个LSTM层的隐层输出h_n作为全连接层的输入。LSTM的使用方法可以参考https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html?highlight=lstm#torch.nn.LSTM\n", + "\n", + "由于LSTM有四个门，因此LSTM的参数量是4倍的Input_size×hidden_size，参数量过多就容易过拟合，同时由于数据量也较少，因此该方案中只堆叠了两个LSTM层。" + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:24.329337Z", + "iopub.status.busy": "2021-11-10T09:25:24.327773Z", + "iopub.status.idle": "2021-11-10T09:25:24.329920Z", + "shell.execute_reply": "2021-11-10T09:25:24.330328Z", + "shell.execute_reply.started": "2021-11-10T08:35:02.875723Z" + }, + "papermill": { + "duration": 0.058538, + "end_time": "2021-11-10T09:25:24.330451", + "exception": false, + "start_time": "2021-11-10T09:25:24.271913", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 构造模型\n", + "class Model(nn.Module):\n", + " def __init__(self):\n", + " super().__init__()\n", + " self.conv1 = nn.Conv2d(6, 32, kernel_size=7, stride=2, padding=3)\n", + " self.conv2 = nn.Conv2d(32, 32, kernel_size=3, stride=1, padding=1)\n", + " self.bn = nn.BatchNorm2d(32)\n", + " self.avgpool = nn.AvgPool2d(kernel_size=2, stride=2)\n", + " self.flatten = nn.Flatten()\n", + " self.lstm1 = nn.LSTM(3456, 2048, batch_first=True)\n", + " self.lstm2 = nn.LSTM(2048, 1024, batch_first=True)\n", + " self.fc = nn.Linear(1024, 24)\n", + " \n", + " def forward(self, x):\n", + " # 转换输入形状\n", + " N, T, H, W, C = x.shape\n", + " x = x.permute(0, 1, 4, 2, 3).contiguous()\n", + " x = x.view(N*T, C, H, W)\n", + " \n", + " # CNN部分\n", + " x = self.conv1(x)\n", + " x = F.relu(self.bn(x))\n", + " x = self.conv2(x)\n", + " x = F.relu(self.bn(x))\n", + " x = self.avgpool(x)\n", + " x = self.flatten(x)\n", + "\n", + " # 注意Flatten层后输出为(N×T,C_new)，需要转换成(N,T,C_new)\n", + " _, C_new = x.shape\n", + " x = x.view(N, T, C_new)\n", + " \n", + " # LSTM部分\n", + " x, h = self.lstm1(x)\n", + " x, h = self.lstm2(x)\n", + " # 注意这里只使用隐层的输出\n", + " x, _ = h\n", + " \n", + " x = self.fc(x.squeeze())\n", + " \n", + " return x" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:24.429647Z", + "iopub.status.busy": "2021-11-10T09:25:24.429062Z", + "iopub.status.idle": "2021-11-10T09:25:24.431608Z", + "shell.execute_reply": "2021-11-10T09:25:24.432056Z", + "shell.execute_reply.started": "2021-11-10T08:35:03.286113Z" + }, + "papermill": { + "duration": 0.057262, + "end_time": "2021-11-10T09:25:24.432197", + "exception": false, + "start_time": "2021-11-10T09:25:24.374935", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "def rmse(y_true, y_preds):\n", + " return np.sqrt(mean_squared_error(y_pred = y_preds, y_true = y_true))\n", + "\n", + "# 评估函数\n", + "def score(y_true, y_preds):\n", + " # 相关性技巧评分\n", + " accskill_score = 0\n", + " # RMSE\n", + " rmse_scores = 0\n", + " a = [1.5] * 4 + [2] * 7 + [3] * 7 + [4] * 6\n", + " y_true_mean = np.mean(y_true, axis=0)\n", + " y_pred_mean = np.mean(y_preds, axis=0)\n", + " for i in range(24):\n", + " fenzi = np.sum((y_true[:, i] - y_true_mean[i]) * (y_preds[:, i] - y_pred_mean[i]))\n", + " fenmu = np.sqrt(np.sum((y_true[:, i] - y_true_mean[i])**2) * np.sum((y_preds[:, i] - y_pred_mean[i])**2))\n", + " cor_i = fenzi / fenmu\n", + " accskill_score += a[i] * np.log(i+1) * cor_i\n", + " rmse_score = rmse(y_true[:, i], y_preds[:, i])\n", + " rmse_scores += rmse_score\n", + " return 2/3.0 * accskill_score - rmse_scores" + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:24.537353Z", + "iopub.status.busy": "2021-11-10T09:25:24.536763Z", + "iopub.status.idle": "2021-11-10T09:25:24.981477Z", + "shell.execute_reply": "2021-11-10T09:25:24.981909Z", + "shell.execute_reply.started": "2021-11-10T08:35:03.765488Z" + }, + "papermill": { + "duration": 0.504348, + "end_time": "2021-11-10T09:25:24.982067", + "exception": false, + "start_time": "2021-11-10T09:25:24.477719", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Model(\n", + " (conv1): Conv2d(6, 32, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3))\n", + " (conv2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))\n", + " (bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n", + " (avgpool): AvgPool2d(kernel_size=2, stride=2, padding=0)\n", + " (flatten): Flatten(start_dim=1, end_dim=-1)\n", + " (lstm1): LSTM(3456, 2048, batch_first=True)\n", + " (lstm2): LSTM(2048, 1024, batch_first=True)\n", + " (fc): Linear(in_features=1024, out_features=24, bias=True)\n", + ")\n" + ] + } + ], + "source": [ + "model = Model()\n", + "print(model)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.044825, + "end_time": "2021-11-10T09:25:25.073027", + "exception": false, + "start_time": "2021-11-10T09:25:25.028202", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "考虑到本次任务的评价指标score=2/3×accskill-RMSE，其中RMSE是24个月的rmse的累计值，我们这里可以自定义评价指标中的RMSE作为损失函数。" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:25.168798Z", + "iopub.status.busy": "2021-11-10T09:25:25.168003Z", + "iopub.status.idle": "2021-11-10T09:25:25.170478Z", + "shell.execute_reply": "2021-11-10T09:25:25.170086Z", + "shell.execute_reply.started": "2021-11-10T08:35:06.590364Z" + }, + "papermill": { + "duration": 0.052236, + "end_time": "2021-11-10T09:25:25.170594", + "exception": false, + "start_time": "2021-11-10T09:25:25.118358", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 采用RMSE作为损失函数\n", + "def RMSELoss(y_pred,y_true):\n", + " loss = torch.sqrt(torch.mean((y_pred-y_true)**2, dim=0)).sum()\n", + " return loss" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:25:25.273264Z", + "iopub.status.busy": "2021-11-10T09:25:25.272445Z", + "iopub.status.idle": "2021-11-10T09:26:21.119405Z", + "shell.execute_reply": "2021-11-10T09:26:21.119950Z", + "shell.execute_reply.started": "2021-11-10T08:37:07.273086Z" + }, + "papermill": { + "duration": 55.904879, + "end_time": "2021-11-10T09:26:21.120168", + "exception": false, + "start_time": "2021-11-10T09:25:25.215289", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch: 1/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 20.67it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 18.423\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.20it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 17.787\n", + "Score: -10.458\n", + "Epoch: 2/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.10it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 16.106\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.00it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 16.873\n", + "Score: -15.903\n", + "Epoch: 3/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.29it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 15.436\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.55it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 16.488\n", + "Score: -28.155\n", + "Epoch: 4/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.25it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 15.036\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 53.23it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 16.916\n", + "Score: -22.716\n", + "Epoch: 5/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.27it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 14.674\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.23it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 16.502\n", + "Score: -23.859\n", + "Epoch: 6/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.22it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 14.167\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.26it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 17.340\n", + "Score: -19.200\n", + "Epoch: 7/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.26it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 13.599\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.89it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 18.158\n", + "Score: -17.133\n", + "Epoch: 8/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.00it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 12.915\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 40.89it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 21.207\n", + "Score: -26.217\n", + "Epoch: 9/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 20.92it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 12.190\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 51.79it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 17.459\n", + "Score: -37.787\n", + "Epoch: 10/10\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 100/100 [00:04<00:00, 21.26it/s]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Training Loss: 11.559\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 52.77it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Validation Loss: 17.482\n", + "Score: -14.426\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "model_weights = './task03_model_weights.pth'\n", + "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n", + "model = Model().to(device)\n", + "criterion = RMSELoss\n", + "optimizer = optim.Adam(model.parameters(), lr=1e-3, weight_decay=0.001) # weight_decay是L2正则化参数\n", + "epochs = 10\n", + "train_losses, valid_losses = [], []\n", + "scores = []\n", + "best_score = float('-inf')\n", + "preds = np.zeros((len(y_valid),24))\n", + "\n", + "for epoch in range(epochs):\n", + " print('Epoch: {}/{}'.format(epoch+1, epochs))\n", + " \n", + " # 模型训练\n", + " model.train()\n", + " losses = 0\n", + " for data, labels in tqdm(trainloader):\n", + " data = data.to(device)\n", + " labels = labels.to(device)\n", + " optimizer.zero_grad()\n", + " pred = model(data)\n", + " loss = criterion(pred, labels)\n", + " losses += loss.cpu().detach().numpy()\n", + " loss.backward()\n", + " optimizer.step()\n", + " train_loss = losses / len(trainloader)\n", + " train_losses.append(train_loss)\n", + " print('Training Loss: {:.3f}'.format(train_loss))\n", + " \n", + " # 模型验证\n", + " model.eval()\n", + " losses = 0\n", + " with torch.no_grad():\n", + " for i, data in tqdm(enumerate(validloader)):\n", + " data, labels = data\n", + " data = data.to(device)\n", + " labels = labels.to(device)\n", + " pred = model(data)\n", + " loss = criterion(pred, labels)\n", + " losses += loss.cpu().detach().numpy()\n", + " preds[i*batch_size:(i+1)*batch_size] = pred.detach().cpu().numpy()\n", + " valid_loss = losses / len(validloader)\n", + " valid_losses.append(valid_loss)\n", + " print('Validation Loss: {:.3f}'.format(valid_loss))\n", + " s = score(y_valid, preds)\n", + " scores.append(s)\n", + " print('Score: {:.3f}'.format(s))\n", + " \n", + " # 保存最佳模型权重\n", + " if s > best_score:\n", + " best_score = s\n", + " checkpoint = {'best_score': s,\n", + " 'state_dict': model.state_dict()}\n", + " torch.save(checkpoint, model_weights)" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:21.428391Z", + "iopub.status.busy": "2021-11-10T09:26:21.427573Z", + "iopub.status.idle": "2021-11-10T09:26:21.430216Z", + "shell.execute_reply": "2021-11-10T09:26:21.429741Z", + "shell.execute_reply.started": "2021-11-10T08:37:59.516491Z" + }, + "papermill": { + "duration": 0.15808, + "end_time": "2021-11-10T09:26:21.430330", + "exception": false, + "start_time": "2021-11-10T09:26:21.272250", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 绘制训练/验证曲线\n", + "def training_vis(train_losses, valid_losses):\n", + " # 绘制损失函数曲线\n", + " fig = plt.figure(figsize=(8,4))\n", + " # subplot loss\n", + " ax1 = fig.add_subplot(121)\n", + " ax1.plot(train_losses, label='train_loss')\n", + " ax1.plot(valid_losses,label='val_loss')\n", + " ax1.set_xlabel('Epochs')\n", + " ax1.set_ylabel('Loss')\n", + " ax1.set_title('Loss on Training and Validation Data')\n", + " ax1.legend()\n", + " plt.tight_layout()" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:21.749325Z", + "iopub.status.busy": "2021-11-10T09:26:21.732622Z", + "iopub.status.idle": "2021-11-10T09:26:22.127478Z", + "shell.execute_reply": "2021-11-10T09:26:22.128264Z", + "shell.execute_reply.started": "2021-11-10T08:37:59.537815Z" + }, + "papermill": { + "duration": 0.549238, + "end_time": "2021-11-10T09:26:22.128511", + "exception": false, + "start_time": "2021-11-10T09:26:21.579273", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS8AAAEYCAYAAAANoXDNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA8x0lEQVR4nO3dd3hUVfrA8e/09B4gAVIgcKR3RUFpigUUaaKiIJafZVfXVdeyu7q6u5Z1de26dgQUEAVFULChIKL0joeWBEJCKEkgdSZTfn/cgY2YQMpM7szkfJ6Hh8ydO/e+Z8p7zzn33HMNHo8HRVGUYGPUOwBFUZTGUMlLUZSgpJKXoihBSSUvRVGCkkpeiqIEJZW8FEUJSip5tVBCiC+EEFN9va6ehBA5QogL/bDd74QQN3v/niyE+LI+6zZiP2lCiDIhhKmxsbYkZr0DaE5CiBzgZinl13rH0hhCiLIaDyMAO+DyPr5VSvl+fbclpbzUH+sGIiHEg8BlUsoLTlmeBOQDfaWUW+uzLe97XO/3+Qxx5VDj+yil3AdE+WLbtezLA1QAHrTvzUbgDSnl3Hq+figwS0rZzh/xNYaqeQURKWXUiX/APuDyGstO/qCEEC3qoFQPs4DzhBCZpyy/GthS38QVAnp5vzsCmA68LIT4m74hNZ76kgNCCBvwL+Aq76IPgQeklHbv0Xk6MBhwA9uAIVJKtxDiAeAuIAbtCH6HlPKbWrYfC7wEXIp29HsTeMK7jRuAm4GfgJuAEu92vmhA/EPRfqAvAX8EvhJC3AXMBM5B+5xXArdJKfO8r/kO7Uj61pliaOC6mcB7QB/gZ0ACsVLK62qJO74eMa4AhgM9gVXAtVLKI97nrwf+iVZb+U9d74+UMk8I8S1wPfD3Gk9NAWacKY5TYr4BrbY02Pv4Iu/7nuLdhqHGuh3RPuteaDWepcDvpJQlQoiZQBrwmRDC5Y3rQyAbsEgpnUKIVOC/aN+9IuBfUso3vdt+FOgKVAFj0Q5mU6WUa+t6H2q8H0eAmUKISmCWEOJlKeVRIcQ04H6gHXDYu7/XhRCRwBeArUbtv7N3vReALkAl8DFwj5TScaYYfEHVvDR/AQYCvdG+aGcDf/U+dy+QByQDrYE/Ax4hhAB+DwyQUkYDFwM5dWz/JSAW6AAMQfvRTKvx/DloP/Ik4GngbSGE4dSNnEEbIAFIB/4P7bN91/s4De3L9fJpXt+QGE637gfAaiAReBQtYdSlPjFei/ZetQKswH0AQoiuwGve7ad693e6Js17NWPxfn69vfE29L06sY0kYD7adyUJ2AMMqrGKAXjSG18XoD3ae4KU8np+XXt+upZdzEH77qUCE4AnhBDDazx/hXedOGBhfWI+xadoyfps7+NDwGi0g/E04DkhRF8pZTnagTe/Rk0/H63L4o/esp8LjADuaGAMjaZqXprJwJ1SykMAQojHgNeBh4FqtKNqupRyN1pNAO/R0gZ0FUIcllLm1LZhb+fr1UBvKWUpUCqEeBbth/S2d7XcGkfU94BX0RLlwQaUwQ38TUpp9z4+cSQ8EcfjwLLTvL4hMdS6rhDCCgwARniPvj8IIRbWtUMp5dF6xPiulHKn9/kP0X6woP2YF0kpl3ufexjtYFKXBcBrQojzpJQ/oh1AvpBSHvY+35D36oTLgG1Syo+8r3se7WB3ony7gd3eh4eFEP8B6tVME0K0R0uEo6SUVcBGIcRb3ri/9a72g5Tyc+/6M4G767PtGvFVCyGOoB30kFIurvH0994TE+cD6+t4/boaD3OEEK+jHZyfb0gcjaWSlyYVyK3xONe7DODfaEfLL7WDNW9IKZ+SUu4WQtztfa6bEGIpWpU5/5RtJwGWWrbftsbjkwlCSlnh3U9DO24Pe7/kAAghIoDngEuAeO/iaCGESUrpquX1DYmhrnWTgCIpZUWNdfej1Th+o54x1kyeFTViSvVu+0Qc5UKIo3XEeyLOecAUIcQqtAPWvQ2IozanxuARQpx8LIRojdasOh+IRqvhFZ9me6duu8h7wDshF+hf4/Gp702YEMIspXTWZwdCCAtai6LI+/hStOTa2RtrBLDlNK/vjNZc7+9d1wysq2t9X1PNRk0+WpPhhDTvMqSUpVLKe6WUHdCO+vcIIUZ4n/vA2/eRjtan8a9atn0ErfZ26vYP+LgMp04Pci9ax+w5UsoY4MSZtoY2RxuiAEjwJoMTak1cXk2JsaDmtr37TDzDa95D69e8CC2ZfNbEOE6NwcCvy/sE2ufSw7vd607Z5ummdMlHey+jayzz9fdmDOAEVnv7fT8GngFaSynjgM9rxFtbrK8BvwCdvOX7M/79fv1KS6x5WYQQYTUeO4HZwF+FEGvQPqRH0DrAEUKMRvuA9gDH0Nr5bm+fSVu0zt0qtGbab8bnSCld3ubO40KIKWhV9HvQviT+FO2NqUQIkUA9mytNIaXMFUKsBR4VQvwV6Adczv+ShC9j/Aj4WQgxGK2P7e+c+WC8Au0EwxvAnBody42NYzHaGbtxaH1Ov0PrezwhGu07c0wI0Rb40ymvL0TrB/0NKeV+IcSPwJNCiPvQakM3odUYm8RbxkvRak3/8nbWR6N1gxwGnN5a2EjgxJnYQiBRCBErpTxWo3zHgTIhxFnA7d7XN4uWWPP6HO2LeuLfo2hnrNYCm9Gqyeu9ywA6AV8DZWhnu16VUi5D+6CfQqtZHUTrUH6ojn3eCZQDe4Ef0DqJ3/FtsX7jeSDcG99PwBI/7++EyWidt0fR3sO5aOOKavM8jYxRSrkNLVl8gFYDKkbr3D7dazzADLRa8IymxuE9azcR7XtwFO27srLGKo8BfdES2GK0zv2ankQ7aJZ4E9SprgEy0GphC9D6NJsyRnGT92zhbrQzxn+UUj7iLUsp2pnzD9Hey2vREvKJsv6CdpDf6403Fe3kybVAKdpZ1XqNGfMVg5qMUPEnIcRc4BcpZdCOJ1ICU0tsNip+JIQYgNYBnI3W7BiDVjNRFJ9SyUvxtTZozaNEtGbc7VLKDfqGpIQi1WxUFCUotcQOe0VRQkBQNBvdbrfH5ap/DdFkMtCQ9QNZKJUFQqs8oVQWCMzyWCymI2gDaX8jKJKXy+WhpKTizCt6xcVFNGj9QBZKZYHQKk8olQUCszzJydG5dT2nmo2KogQllbwURQlKKnkpihKUgqLPqzYul5Pi4sM4nb+d96yw0ECoDAFpjrKYzVbi45MxmYL266C0QEH7bS0uPkxYWASRkW0wGH59IbvJZMTlcusUmW/5uywej4fy8uMUFx8mKSnFb/tRFF8L2maj0+kgMjLmN4lLaRiDwUBkZEytNVhFCWRBm7wAlbh8RL2PSjAK6uSlKKHOXLgBQ1V9J19tWVTyUpQAZSwvJG7+WCJX/vPMK7dAKnk1UmlpKfPnz2vw6+677y5KS0vPvOIpHn/8UZYtC8p75SqNFLZjDga3E1v2EnCpPslTqeTVSGVlpSxY8Nvk5XSe/t4HzzzzItHR0addR1Fwuwjb9j7usHiM9mNY837QO6KAE7RDJWpavK2QhVv/dyMVgwGaOjTqiu5tGNWtdZ3P//e/L3HgwAFuuOFazGYzVquV6OhocnNzmTNnPg89dC+FhYU4HA4mTryaMWPGATBhwuW89dZMKisruO++u+jZszdbtmwmOTmZp556FpstrM59nrB27WpeeeV5XC4XZ53Vlfvuewir1cprr73EypXLMZlMDBgwkN///m6+/fZr3n33DYxGE1FRUbzyyptNe2OUZmHN/RZTWT7HR75C1HcPYt29GEf68DO/sAUJieSlh9tuu5O9e/cwffoHrF+/lvvvv5sZM+aSmqrd0eyhhx4hJiYWu72Km2+ewtChw4mNjfvVNvLy9vPoo4/zwAN/5eGHH+S7777l4osvO+1+7XY7TzzxGM8//yppaen84x+P8MknH3HxxZexfPkyPvjgYwwGw8mm6fTpb/Kf/7xMcnKrRjVXFX2EbZ2BK7I19o6jsOZ+iy17CWWup8Bk0Tu0gBESyWtUt9a/qiXpMUi1S5duJxMXwLx5c1i+/DsADh0qZP/+/b9JXikpqXTqJAAQ4iwKCk695eNv7duXS0pKKmlp2p3ULr10NPPnz2PcuKuwWm08+eTfGTTofM4773wAevToxeOPP8rw4RcxZMgwH5RU8Tfj8X1Y931HxYC7wWjG3nE0YfJjLHk/UJ2uPsMTVJ+Xj4SHh5/8e/36taxdu5rXX3+X996bTadOAofjtzfQsVj+dxQ1Gk24XKe7v+npmc1m3nzzPYYNG8HKlSu49947AfjTn/7MLbfcwaFDhdx00/UcO1bS6H0ozSN82/tgMFLV9RoAHGkX4LZGY9uzSOfIAotKXo0UERFBRUXtcx+Vl5cRHR1DWFgYubk5bN++tdb1GiMtLZ2Cgnzy8rQbMy9d+jm9e/eloqKC8vIyzj13MHfddS+7d+8C4MCBPLp1687NN99GXFw8hw4V+iwWxQ9cdsJ2zMGRcSHuKO9N2002HBkXYdu7FFzV+sYXQEKi2aiH2Ng4evToxfXXX4XNFkZCQsLJ58455zw++WQ+kydPIC0tna5du/tsvzabjT//+W88/PADJzvsr7xyPMePH+ehh+7B4XDg8Xi4884/AvDKKy+Ql7cPj8dDv35nk5XV2WexKL5n27sEY+VRKrtf/6vl9qzRhO2cj+XASqrThuoTXIAJihtwVFe7PKfO8HjwYC5t2qTXur66MLvhTvd++lIgztbZWP4oS+yC8ZjKDlJ03Qow1GgYOatIfKc39qxRlA1/1qf7PCEQP5vk5Oh1QP/anlPNRkUJEKajEmv+z1R2u+7XiQvAHIYjUzUda1LNxgDz7LP/YsuWTScfGwwwYcLVjBp1hY5RKc0hbNssPEYrVV2uqvV5e8fRhO1coJqOXip5BZh7733gV49DqQmsnEZ1BWHyI+xZo/CEJ9a6iiNtCG5LFLbdi1TyQjUbFSUghO36FKOjlMpu19e9kjkMR8aF2PYuUU1HVPJSlIAQtnUmzgSBM2XAadezZ43GaC/BcuDHZooscKnkpSg6MxduxHJ4szY84gwTQ2pNx0g1YBWVvBRFd2HbZuIxR2AX48+8sjlcNR29VPJqJhdddH6dzxUU5HP99bWfYVJCm6GqhLBdn1LVeSwea/2mSrJnXY6xqhhL/io/RxfYVPJSFB2FyY8xOKuo6n6ajvpTnGw67m7ZTceQGCph++UjwnbMOfnYYGj6vQ6rulyN/awJdT7/2msv0apVa8aP12pMb7/9OiaTiQ0b1lFaehyn08ktt9zO+ecPbdB+7XY7zz77FL/8sh2TycQf/nAvvXv3Y+/ePTz55GNUVzvxeNz8859Pk5SUzCOPPMihQ4dwu13ccMPNjBgxsinFVpqTx0PYtllUt+6DM7kBl5DVaDqWDXkCjCHxM26wlllqHxgx4iJefPE/J5PXsmVf8+yzLzFx4tVERkZRUlLCrbfewODBQxp0d54TU0vPmDGX3Nwc7rnn93zwwcd8+unHTJx4DSNHXkp1dTVut4tVq1aSlJTMv//9AgBlZWW+L6jiN5b8VZiLd3F8+H8a/Fp71mjCdn2K5cAqqtvX3SURykIiednPmvCrWlJzDOzs3PksiouLOHLkMMXFxURHR5OYmMSLLz7Lpk0bMBiMHD58mKKioyQmJtV7u5s3b2TChEkApKdn0KZNG/bv30e3bj2ZMeMdDh0qZMiQ4bRvn0aHDlm8/PLzvPrqiwwadD69evXxV3EVPwjbOgu3LRZ7p8sb/FpH2lBv0/Ezlbx8TQjRHpgBtAY8wBtSyheEEAnAXCADyAGuklIG5b2dhg27kGXLvqGo6CjDh4/kyy+/oKSkhLffnoXZbGbChMtxOHxz44SRIy+hW7fu/PjjD/zpT3/gT3/6M/36DeCdd2axatVK3nzzNfr1G8C0abf4ZH+KfxkqDmPb+wWVPW4Ac/gZ1/8N1XT0a4e9E7hXStkVGAj8TgjRFXgQ+EZK2Qn4xvs4KA0ffhHffPMly5Z9w7BhF1JWVkZ8fDxms5n169dy8GBBg7fZq1dvvvzyC0CbNbWw8CBpaekcOJBHampbJk68msGDh7Bnzy6OHDmMzRbGxRdfxjXXXM/Onb/4uoiKn4TtmIvBXU1Vt+savQ17x1EYq4qwHGiZZx39lq6llAVAgffvUiHEDqAtMAYY6l3tPeA74IFaNhHwOnToSEVFOcnJySQlJTFy5KU88MAfmTJlEmed1ZX09IwGb3Ps2Ik8++xTTJkyCZPJxF//+hhWq5Vvv/2apUs/x2w2k5CQyJQp09ixYzuvvvoCBoMRs9nMffcF7XGgZXG7CN82C0fbQbjiOzZ6M470YXjMEdq1ji2w6dgs83kJITKA5UB3YJ+UMs673AAUn3hcF7fb7XG5fh2nlL+Qmprhh2hbpvz8HIQ4y+/7CaULzRtbFsPurzDPnYRz3Dt4ulzZtBgW3IQhZznOP+xoctMxED8bi8VU53xefm8oCyGigI+Bu6WUx4UQJ5+TUnqEEGfMni6X5zeTpHk8njrf6ED8EBqrucri8fz2PfaHQJzwrrEaW5aYn9/EENGK4lbDoInvhTXtEmK3L6B827dUtx/cpG0F4meTnFz3wF2/Ji8hhAUtcb0vpZzvXVwohEiRUhYIIVKAQ/6MIZDs2bObf/zjkV8ts1gsvPnmezpFpDQ34/E8rDnfUNHvTp/cxsyRNhyPORzbnkVNTl7Bxp9nGw3A28AOKWXNgSwLganAU97/P23sPjweT4PGUOmtY8cspk//QO8wfiMYpgIPFWHbPwCDgapuk32zQUs49owLse39grIL/tmizjr6s6SDgOuBLUKIjd5lf0ZLWh8KIW4CcoFGXdRnNlspLz9OZGRMUCWwQOPxeCgvP47ZbNU7lNDnchC+fTaO9BG4o9ueef16snccRdjuz7Dk/0x1u0E+226g8+fZxh+AurLKiKZuPz4+meLiw5SVlfzmOV9cHhQomqMsZrOV+Phkv+5DAWv2lxgrDzdpeERtHOkjtKbj7kUqeQUDk8lMUlJKrc8FYsdjY4VSWVq68K0zcEW3x+HrKZx/03Q0+Xb7AUrNKqEozcBUvBvrgR+p7DbZL8nF3nEUxsojWPJ/8vm2A5VKXorSDLQ7A1mo6jLJL9s/2XTcs9gv2w9EKnkpir9VVxL2yzzsHS7FE+GnvkVLOPb0Edj2fA5ul3/2EWBU8lIUP7Pt/gyj/ViDJhxsDHvW6BbVdFTJS1H8LHzrDJzxnahOHejX/TjSh+Mxh7WYpqNKXoriR+bDW7Ac2qgNj/D3eERLBI70Edj2fNEimo4qeSmKH4VtnYnHHEbVaaYU9yV7x9EYKw9jKfi5WfanJ5W8FMVPDPbjhO1cQFWnMXhssc2yT3vGCK3puDv0m44qeSmKn9h2zsfgrKSq+5Tm26klAkf68BZx1lElL0XxB4+H8K0zqU7uibNVr2bd9f+ajqubdb/NTSUvRfEDc8EazEXS78MjamNPH4HHZMO2J7Tv66iSl6L4QfjWGbitMVR1GtP8O7dG4sgYgTXEzzqq5KUoPmaoPIptz+dUifFgidAlBnvH0ZgqDmE5uEaX/TcHlbwUxce0OwM5dGkynnCy6bg7dJuOKnkpii953IRvex9H6jm4EjrrF4c1Ekf68JBuOqrkpSg+ZNm/HNPx3OYdHlEHe9ZoTBWFIdt0VMlLUXwofOtM3OGJ2Dtcqnco2NMvDOmmo0peiuIjxtJ8rDlfUdXlajAFwD0BrJE40odpTUdPaNwKsCaVvBTFR8K2fwAejzZbaoDQzjoWYikIvaajSl6K4guuasK2z8aRNhR3TJre0ZzkyNCajtYQbDqq5KUoPmDN+QpTRWFAdNTX5LFG4Ugbim3v5yHXdFTJS1F8IHzrTFxRqTjSh+sdym/Ys0ZjKi/EXLBW71B8SiUvRWkiU8lerHkrtLtgB+BtxxwZF4XktY4qeSlKE4Vtex+P0aydZQxAJ5uOexaHVNNRJS9FaQpnJWE75uLIvBh3ZGu9o6nTyabjwXV6h+IzIZW8yuxOHvxsO7lHy/UORWkhbHsWY7SXUBlgHfWnOtl0DKGzjiGVvAwG+CmnmKeWSL1DUVqI8K0zccZ1oLrteXqHclqh2HQMqeQVaTUzZUB7vv7lEJsOHNM7HCXUFW7FcnAdVd2u9/+dgXzA3nEUpvKDmA+u1zsUnwip5AVwTb+2JEfZeGl5Nh6PR+9wlBBmXP8uHpOt2e4M1FSOzBNNx8/0DsUnQi55hVtM/H5YRzblH2fF3iK9w1FClMFRhnHrPOydrsATFq93OPXisUbjaD8kZJqOIZe8ACb2a0dafDgvr8jG5Va1L8X3bPJjDI4yKrvpN+FgY9izQqfp6LfkJYR4RwhxSAixtcay3kKIn4QQG4UQa4UQZ/tj3xaTkTsGZ5B9tILF2wv9sQulpXJVE7HmeaJ+eBR3Sl+crfvoHVGDODIuwmO0hsSAVX/WvKYDl5yy7GngMSllb+AR72O/GN4pia5tonl9ZQ5V1aE5k6TSvMyHtxI/bxSRq5/B3uFSXJNmB0VHfU0eWwyOtNBoOvoteUkplwOndjp5gBjv37FAvr/2bzAYuPP8TA6VOZi30W+7UVoCl52In54mbt4oDJVHOHbpm5Re/CpEJusdWaPYs0ZhKivAXLhB71CaxNzM+7sbWCqEeAYtcfp1cEz/tDjOzYhn+ur9XNkjheiw5i6uEuzMB9cT/e29mIt3UXXWRMoGPRI0HfR1cWSM1JqOuxfhbNNP73Aarbl/zbcDf5RSfiyEuAp4G7jwTC8ymQzExdX/FlImk/Hk+g9e1oUxr/7I3M0F3DdSNDJs/dQsSygImvJUV2D8/kmMq1+DqDY4J83FlHURsTVWCZqy/EYEng7DCM/+HMuoJ8GgNcCCrTzNnbymAn/w/j0PeKs+L3K5PJSUVNR7J3FxESfXTw03c0mXVkxflcsVXVrRKtrWwJD1VbMsoSAYymPJ/4mob+/DdCyHym7XUX7eX/BYo+GUuIOhLHWxpV9KzO6llMmVJ2tfgVie5OToOp9r7qES+cAQ79/DgV3NsdPbBqXjcnt4Y1Vuc+xOCVaOcqKW/4W4BRMweNyUjJlL2dCntMQVYhyZ3rOOuxfrHUqj+XOoxGxglfanyBNC3ATcAjwrhNgEPAH8n0936qwicuU/oHDrrxa3jQ1nfK8UPtt6kJyjgXVkUQKDZf8KEuaMIGzLDCp63kjRpK+objdI77D8xmOLxZF2gfesY3COhfRbs1FKeU0dT/mth9DgdmLb9SnGLdMJG/QwVd2nnjyVfdPANBZtK+SVH7L595hu/gpBCTIG+3Eif/wH4dtn44zrQMnYj3Gm+mX4YcCxdxyNLedrzIUbcLbpq3c4DRZSI+w91iiKJy3Fk3EB0cv/SsySWzBUFQMQH2Hluv7t+G73UTbnH9c5UiUQWHO+Jn72cMJ2zKWiz+0UT1raYhIXnGg6WrTaVxAKqeQF4AlPxDVpNmWD/oY15xvi516M2Xvbp2v7tSMhwsLLy/eqi7ZbMENVMdFf3UXs4hvw2GIpGb+Q8vP+AuZwvUNrVh5bLI72F2hzfAXh7yHkkhcABiOVvW+hZPwnYLQQt2ACEWtfJMIMN5+bzoYDx1mZrS7abomsexaT8MEwbLsXUt7/boqv+hxn6956h6Ube9ZoTGUHgnLAamgmLy9nq14UT1qCPWs0kT8/TezCaxnfwUC7uDBeWZGjLtpuQQwVh4lZciuxS27FFdmG4omfU3HOfWAKrqEzvubIHBm0TceQH3LusUZTetHLONpr/WDJH13CP8SjTPs5kSU7DjGqW+DOO674gMeDbecColY8gqG6grKBD1LZ+1YwWfSOLCBoTcfzse1ZjNvzxClPerTrH93VGNxOcFeD24XBXQ1up/d/V43nnTWWO2ssq8bg0dZzxaTjTBngk9hDPnkBYDBg7zIJZ+u+xHx5B8M23ckzMWN5/odJXCiSsZlDugLaYhnLCoj6/iFsOV9T3bovpcOfxZXQSe+wAo6942hsufdgfC6LRNf/ko7BXe3zfbmiUimauton22oZycvLldCJ4gkLiVr5TyZsfY8s9ya++vnfjB50jt6hKb7k8RC2Yw6RK/+OwV1N2aC/UdnzxoC8p2IgsGddTkWRxGZyYa/2gNEMRgseo8n7v/YYowmP0QJGs3fZr9erufzEev9bZsFjMOGJSPJZ3C0qeQFgDqdsyOM42g0ia+k9ZG28jor4pzF0Hat3ZIoPGI/nEf3d/Vj3L8eROpDSYf/GHZepd1iBzRJO+aCHscRFUB5glwedTottLzk6Xsb2Sz7hF3d7Wi27k6hlf4LqSr3DUhrL4yFs+wfEz7kQS8FaSi94nGNXfqgSVwhrsckLILPDWbyV8SKvua8kbPsc4uddhunoDr3DUhrIWJZP7KLriF52P85WPSm65huqekw9OVuCEppa/Kd76/kdecY5if+2/TcG+zHi540mbOvMoBy01+J4PNh2fEj87Aux5K+m9IJ/cmzMHNwx7fWOTGkGLT55tYsLZ3zPFJ7Zm8rmiz+luu1Aor9/iJilt2Kwq3s/BipjeSExn08j5tt7cCZ2oejqr6jqcYOqbbUg6pMGbhyYhtVs5OV1pRwbPZOyc/+CNftL7dKig+v0Dk+pyTtuK372cKz7V1A2+FGOjZ2HOzZD78iUZqaSF5AYqV20/c3OI2w9WEZl39spGTsfMBA3fxzh614O+psVhAJDxRFiltxCzFd34orrSPGkL6nsdbOqbbVQ6lP3mty/HfHhFl5eod1p29mmL8WTlmLveBlRPz1F7MLJGMoP6R1mi2XdvYiE2cOx5nxL2bl/pmTcAlzxHfUOS9GRSl5ekVYzNw1MY93+Y6zK0abR8dhiKB35KqXDnsZycA0Jc0di2fe9zpG2LIbKIqKX3kHs0ttwxbSneNISKvveoQacKip51TSuVwqpsWG8vCIb94mzjQYDVV2vpXjCYtzhicR9NpnIVU+Ay/eXTjQ7txPL/hVELf8r4etexngsR++IfsW6dwkJs4dj2/sF5ec8QMn4T3EldNY7LCVAtLwR9qdhMRm5Y1AGf/38F5bsOMRlXf930bYrUVA8YRFRKx8jYv2rWPZ9j/2sidgzR+KOSdMx6gbyuDEXrCVs96fYdi/GWHkEjzkMg7OKqJ+eojqpO/as0dg7jtJtgKehqoSoFY8QtnM+1UndKL3iA1xJXXWJRQlchmCYlK+62uVp7N2DGsrt8TBl1gZKq6qZN20A1lou2rbuXkTkmucwF0kAnIlnYc+8GEfmxTiTe/j0Lso+uaOLx4P58GZsuxZi270QU1kBHpMNe8ZF2DtdgSN9GMbKImx7FmPb/RkW79xO/khkZyqPNecbopbdj7HqKBX97qSi310BOwNEIN5tpykCsTzJydHrgP61PaeSVy1+yinizo+3cs+wjlzTt22d6xmP5WDL/hJr9lIsBWsweNy4olJwZIzEnjmS6rbngsna6DigaWUxHf0F266FhO36FNPxXDxGC460oVrCyrgIjzWq1tcZSw94E9kiLIXrAahO6oaj42jsWaNwxXXweXkM9uNE/vAY4b/MxZkgKL3wee1AEMAC8cfeFIFYniYnLyFEJFAppXQLIToDZwFfSCmbpeOnuZOXx+Phdx9tYdfhchbcNIAo25lb14bKIqy532DLXop13/cYnJW4rdE40obhyByJI304HltMg2NpaFlMJXu1GtauhZiLd+IxmKhuN5iqTlfgyLwYT1hcg/avJbLPse1ZhMU75s2Z2BV71uWNSmS1lcey73uil92HsbyQir6/o2LA3UExSWAg/tibIhDL44vktQ44H4gHVgJrAIeUcrIP46xTcycvgO0HS5n6/gZuGpjGbYMyGvZiZyXWvJVYs5diy/5K61cymqlOPRd75kgcmRfjjk6t16bqUxbj8Txsuz/DtnshlsNbAHCknoO90xjsHS7z2TQkxtJ8rUb2m0Q2GnvW6HolsprlMTjKiFz5D8K3v48zvhOlI54LqimZA/HH3hSBWB5fJK/1Usq+Qog7gXAp5dNCiI1Syt6+DbV2eiQvgIc+28EPe4+y4OazSYpsZPPP48ZcuAHb3iVYs7/EXLIHgOrkHjgyR2LPvBhXYpc6+8nqbGaVH8K2ZxFhuxZiObhW22ar3lrCyhqFO6p+ybGxjKX52PZ+rjUtvft3Jnbx1sjqTmQnymPJW0n0t/diLD1AZZ9bKT/7PjCH+TVmXwvEH3tTBGJ5fJG8NgB3AM8BN0kptwkhtkgpm6VTQq/kta+4kqumr2VsjzY8cKFvZuA0Fe/x1si+xHxwHQY8uKLbY8+8CEfmxVSnnP2rDupf1VSqirUm3K6FWPJXYfC4cSZ2oarTGOxZl+OOTfdJjA1VdyIbjb3j6F8NJo2L8OBc8lfCt7yHMzZTq22l1PrdDHiB+GNvikAsjy+S1xDgXmCllPJfQogOwN1Syrt8Gmkd9EpeAE99vYtPthzkwxv6kxbv21tjGSoOY8v5Gmv2l1j3L8fgsuO2xeJIH6F1+KcNJTY2jMoNn2DbvVBbx+3EGdcBe9YV2DtdEXDjnoxl+d4+ssVYvLeccyaehb3jaJwJnYn56QkoyaWy102Un/MAWIL3dmOB+GNvikAsj0/PNgohjECUlLLZ7tyqZ/I6Uu5g7FurGdwhkScv7+KTbdaqugLr/u+1s5c5X2OsKsZjtILBgMFlxxXdDnunK6jKGqONefLhcAx/qS2ReeIyODbsGapTB+ocXdMF4o+9KQKxPL6oeX0A3Aa40DrrY4AXpJT/9mGcddIzeQH8d2UOb/+0j/cm96Frm2ifbbdObieWg2uxZn+FzWrkePtLcLbuGxQJqy7GsgLMhzYS0f1iSipC48KOQPyxN0Uglud0yau+36Ku3prWlcAXQCZwvU+iCwLX9W9HXLiFl7wXbfud0Ux16kDKBz2M+6LHcbbpF9SJC8AdlYKjw6VQx9gyRWmo+iYvixDCgpa8FnrHdwX+6FYfibKZuXFgGmv3lfBzbrHe4SiKQv2T1+tADhAJLBdCpAPN1ucVCMb3TCE1xsbLK3L+d9G2oii6qdeF2VLKF4EXayzKFUIM809IgclqNnLroAz+9oXkq18Oc3GXVnqHpCgtWr2SlxAiFvgbcIF30ffA34EWNcn7JV1aMWttHq+tzGF45yQsptDoeFaUYFTfX987QClwlfffceBdfwUVqIwGA78/P5MDx6pYsLlA73AUpUWr73xeHaWU42s8fkwIsfF0LxBCvAOMBg5JKbvXWH4n8Du0YReLpZT3NyxkfZ2bEU+/9rG8/mMuEVYTl3ZpjckY3GcCFSUY1bfmVSmEGHzigRBiEHCm20tPBy6pucDbTzYG6CWl7AY8U/9QA4PBYOChCzvRNjaMx5bsZPLMdazYc7R5hlAoinJSfWtetwEzvH1fAMXA1NO9QEq5XAiRccri24GnpJR27zpBeUeL9IQI3pvch292HuG1lTnc88k2+rSN4fcXdKBnasOnvVEUpeEadHmQECIGQEp5XAhxt5Ty+TOsnwEsOtFs9DY1P0WrkVUB90kp15xpv2632+Ny1T9Ok8mIy9U8tyqrdrmZty6Pl5bt5kiZg4u6tOKeCzuT1co3gzGbsyzNIZTKE0plgcAsj8ViqnOEfYPmsD/lesZ7gOcbGIsZSAAGAgOAD4UQHaSUp81MLpenQZctNPdlDpd1TmJoRjyz1+cxc00e3/zyA5d3a8Mt56XTOrppk+oF4iUbTRFK5QmlskBglic5ue7L8Zpyrr8xvdR5wHwppUdKuRpwA76ZKU9nEVYTNw1MZ8FNA5jUpy2Ltxcy/p01vLQ8m+NVIXCnIUUJME1JXo3pof4EGAbgnU7aChxpQgwBJz7Cyj3DOvLRjf0Z3imJmWv2M/btNcxcsx+7M7Cq5IoSzE7b5yWEKKX2JGVAm1G1zmanEGI2MBStZlWINsh1JtqYsd6AA63P69szBan3rBJNsfNQGa/8kM2P2cW0irJy66AMRnWt//CKQCqLL4RSeUKpLBCY5VF3DwoAa/eV8NKKbLYfLKVDYgR3DM7kgo4JGM4wW0QglqUpQqk8oVQWCMzy+GJKHKWJ+qfFMf3a3jx1eRecbg/3fbqNW+ZsYtOBFnWFlaL4jEpezchgMDCiczJzp/bjoQuzyDtWxc1zNnHvJ9vYe7Rc7/AUJaio5KUDs8nIuF6pLLhpAHcMzmDd/hKueW8d/1gqKSy16x2eogSFBo3zUnwr3GJi2jlpjO2Rwrur9zFvYz5LfznMpD6pTD27PTFhgXmbe0UJBKrmFQDiIiz8cWhHPpo2gBGdk5i5Jo8r31rDjNX7qap26R2eogQkdbYxAO06XMYrK3JYmV1EdJiZwZkJDOuUxLkZ8YRZTHqH1yTB/tnUFEplgcAsz+nONqpmYwDqlBzF8+O6syHvGEt3HeHr7YV8seMQNrOR8zITGN4picEdEoiyqY9PabnUtz+A9WkXy7DuKRwZ0oENeSV8u/MI3+0+yrJdRzAbDZydHsewrCSGZCUSH2HVO1xFaVaq2RjgTi2L2+Nha0Epy3Yd4dtdR8g/VoXR4E10WUkM7ZTU5IvB/SmUP5tgF4jlUSPsg9jpyuLxeNh5uJxlu46wbNcR9h7V1uueEs2wrCSGdUqifXx4c4Z7Ri3lswlGgVgelbyCWEPKklNUcTKR7SgsA6BTcuTJRNYxKeKMlyP5W0v9bIJBIJZHJa8g1tiyFByvYtmuI3y36wgbDxzHA7SPC2NYp2SGd0qka5toXRKZ+mwCVyCWR51tbIFSYsK4tl87ru3XjqPlDr7fc5RlO4/w/ro8ZqzZT6soK8M6aTWy3m1j1U1ElKCjklcLkBhpZVzPFMb1TOF4VTU/7C3i251H+GTLQeZuyCc+3MKgDgkMykzgnPR4osPU10IJfOpb2sLEhFm4rGtrLuvamgqHi1U5RSzbdYTle46yaFshJgP0ahvLoMwEzuuQQMdE/fvJFKU2Knm1YBFWEyM6JzOiczJOt4dtBcf5YW8RK7OLeGlFNi+tyKZNtI1BHRI4LzOBAWlxhAf5CH8ldKjkpQBgNhro1TaWXm1j+d35mRwqtfNjtpbIPt9eyMebCrCaDPRtH8egTK2JGWjDMJSWRZ1tDHCBUBaH083GA8dYmV3Eyr1F5BZr9xtOiw/nvMwEBmcm0KddLFbzma/zD4Ty+EoolQUCszxqqEQQC8Sy5JVU8mN2ET/sLWLd/hIcLg/hFiMD0uIZlBnPeZkJtIkJq/W1gViexgqlskBglkcNlVB8ql1cOFf1actVfdpSVe1i7f4SVnr7ypbvOQpAVlKkVivrkECP1BjMaiiG4mMqeSlNEmYxMbhDIoM7JOLxeMguqmDl3iJ+zC46OaYsymZiYLqWyMb0b693yEqIUM3GABfMZSmzO1mdW6z1lWUXc7TcQaTVxCVdWjGuZwqdW0XpHWKTBPNnU5tALI9qNiq6iLKZGd45meGdk3F7PGzJP85ieZhFWw7y8aYCeqTEML5XCiM6JwX9JItK81M1rwAXSmUBrTy5BcdY7B1+sa+4ktgwM6O6tWZczxTSEyL0DrHeQvGzCbTyqJqXElBiwy1c268d1/Rty7r9x/h4Uz5zN+TzwboD9E+LY0KvFIZ0TMRsUrdYUOqmkpeiG4PBQP+0OPqnxXGk3MFnWw8yf1MBD362g8RIK2O6t2Zsz5Q6h10oLZtKXkpASIq0Mu2cNKYMaM+qnCI+3lTAuz/vZ/rq/ZyXmcCEXqkMzIhXs18oJ6nkpQQUk9FwcuhFwfEqPtlcwCdbDnL33q2kxNgY2zOFK7q3ITFSzdnf0qkO+wAXSmWBxpXH6XLz/Z6jfLSpgLX7SjAZDQzLSmJ8rxT6tY/VbdYL9dn4n+qwV4Ka2WQ8OftFTlEFCzYXsGhbIV/vPEx6fDjjeqUwultrdYfxFkbVvAJcKJUFfFeeqmoX3+w8wsebCthScByb2chFIpnxvVLo1kxTXKvPxv9UzUsJOWEWE6O6tWZUt9bsPFTG/M0FfLH9EIu2FXJWqyiu7d+Wizonq+EWIcxvNS8hxDvAaOCQlLL7Kc/dCzwDJEspj5xpW6rmFRplAf+Wp9zhZMmOQ8zdkE/20QpaR9u4pm9bruzZhkir74/T6rPxv9PVvPx5WJoOXHLqQiFEe2AksM+P+1ZaoEirmfG9UpkztR/Pje1G29gwnv9+L6Pf+JmXlmdzuMyud4iKD/kteUkplwNFtTz1HHA/EPidbUpQMhq04RavT+rF9Ml9GJiewKy1+7nizdU8tkSy50i53iEqPtCsfV5CiDHAASnlJiFEvV9nMhmIi6v/NW8mk7FB6weyUCoLNH95BsVFMOis1uwrqmD6jzl8tP4Ai7YVMqRzMjcPyuCczIRGd+6rz0ZfzZa8hBARwJ/RmowN4nJ5GtQWD8S2e2OFUllAv/LEGOGuwRlM6deWjzfl8+GGfK5/dw1dWkdxXf92DO+c3OAJE9Vn43/JydF1Ptecp2I6ApnAJiFEDtAOWC+EaNOMMSgtXFy4hZsGprPwlnN46KJOlDtc/GXxL4x/ezVz1h+gwuHSO0Slnpqt5iWl3AK0OvHYm8D61+dso6L4ms1sZFzPFK7s0YYVe44yc00ezy7bw5urcpnQK4WJfdqSpC5BCmh+S15CiNnAUCBJCJEH/E1K+ba/9qcojWE0GBiSlcSQrCQ25x9n1to83v15P7PW5nFp19Zc168dGYnB0w/UkqgR9gEulMoCwVGefcWVfLAuj0XbCrE73ZzfIYHrB7Snd9uYX3XuB0NZGiIQy6PXOC9FCUpp8eE8eGEnFt5yNrecm8bm/OP839xN3Dh7I9/uPIzLHfgH/JZAXR6kKHVIiLDyf+dlMGVAez7bVsgH6/J44LMdtIsL49p+7bjuvEy9Q2zRVM1LUc4gzGJiYu9UPpo2gH9d3oW4cAtPf7ObEc8tZ8HmApyqJqYLlbwUpZ5MRgPDOyfzzjW9eX1ST9onhPPEV7u4dsY6Vuw5SjD0H4cSlbwUpYEMBgN928Ux5+ZzePqKrrjcHu75ZBu3z9vM9oOleofXYqjkpSiNZDAYGNYpiblT+3H/iCz2Hqlg6vsb+MuiHRw4Vql3eCFPddgrShOZTUYm9k7l0i6tmLlmP++vO8Cy3UeY2DuVG89JIzZczfDqD6rmpSg+EmUzc/vgTObfOIBLu7Ri9roDjH17DTPX7MfudOsdXshRyUtRfKxVtI2HLxZ8MKUfPVKjeXF5NhPfXcMXOwpxq059n1HJS1H8JCs5khfG9eCVCT2ICbPwyOeSqbM2sGZfsd6hhQSVvBTFz85Oj2fGdX147FJBSWU1d8zbwh/mb2G3mhSxSVTyUpRmYDQYuKxraz66cQB3XZDJ5vzjTJ6xjn8u3cmhUjU9dWOo5KUozchmNnL9gPYsuOlsru7blsXbCxn3zhpeW5lDucOpd3hBRSUvRdFBXLiFPw7tyLxp/RnSMZF3ftrH2LfWMG9jPk6XOjNZHyp5KYqO2sWF8/joLky/tjcZiRE8/c1uJr23jmW7jqjLjc5AJS9FCQDdUmJ4/aqePHtlN4wGuH/hdm6Zs4nN+cf1Di1gqeSlKAHCYDBwQcdEZk/tz0MXZrG/pJKbZm/kz4t2cPB4ld7hBRx1eZCiBBiz0cC4Xqlc0qU1M9fsZ+baPJbvOcrUAe25fkA7wiwmvUMMCKrmpSgBKsJq4tZBGcyb1p/zOyTwxqpcJr67lq/lYdUfhkpeihLwUmLCePLyrvz3qp5Eh5l5aNEObvtwM7sOl+kdmq5U8lKUINGvfRwzruvLgxdmsedIOdfNXM9TX++ipKJa79B0oZKXogQRs9HA+F6pfHzjACb2TuWTzQWMf3cNc9cfaHHTUavkpShBKDbcwn3Ds5g1pR+iVRTPLNvD5BnrWJ3bci76VslLUYJYVlIkr0zowb+v6EqV083vPtrCnz7d1iJmclXJS1GCnMFgYGinJD68oT93DM7g59xirnp3La/+kE2Fw6V3eH6jkpeihAib2ci0c9L4aNoARnRO5t2f9zPBOwliKA6tUMlLUUJMq2gbf7/sLN66uhdJkVYe+Vxy85xN7CgMrTsbqeSlKCGqV9tYpk/uw8MjO5NXUsnUWRv459KdFFU49A7NJ1TyUpQQZjQYuKJHGz6+cQCT+7fT5g97ew2z1uZRHeRT76jkpSgtQJTNzB+GdGDO1H70bhvLC9/v5Zr31rEyu0jv0BpNJS9FaUHSEyJ4flx3nh/bHQ9w9/yt/HHBVvYVB9/QCpW8FKUFGtQhgTlT+/GHIR3YkHeMSdPX8rfPtlEYRPPpqylxFKWFspiMXNe/HZd2acWbq3KZty6PeevyGNczhalntyc5yqZ3iKdl8Nf4DyHEO8Bo4JCUsrt32b+BywEHsAeYJqUsOdO2qqtdnpKSinrvOy4ugoasH8hCqSwQWuUJpbIAlGPg+S8ln20r9F5DmcKUAe1JjLTqFlNycvQ6oH9tz/mz2TgduOSUZV8B3aWUPYGdwEN+3L+iKA3QNi6cv4zszEfT+jNSJDN3/QHGvLWaF7/fS3EADq/wW/KSUi4Hik5Z9qWU8sT9nX4C2vlr/4qiNE67uHAeuUQwb9oALuycxPvr8hjz1mpeXpFNSWXgTL+jZ5/XjcDc+qxoMhmIi4uo94ZNJmOD1g9koVQWCK3yhFJZ4LfliYuL4PnMRO46XMbL3+1hxpr9fLQxn6nnZnDjoAxiwy06RqtT8hJC/AVwAu/XZ32Xy9OgvoVQ6osIpbJAaJUnlMoCdZcnwWLkkYs6cV3fVN78cR+vfr+H91blMLlfO67p15Yom//SSHJydJ3PNftQCSHEDWgd+ZOllKF3taiihKgOiZE8eXkXZk/pxznp8byxKpcr3lzN2z/lUmZv/rt9N2vyEkJcAtwPXCGlDJ1DlqK0IFnJkfzriq7Mur4vfdvF8t+VuVz51mqm/7yvWafg8edQidnAUCAJKAT+hnZ20QYc9a72k5TytjNtSw2VCI2yQGiVJ5TKAo0vz/aDpby5Kpcf9hYRF25hyoB2TOidSrgPbtF2uqESfktevqSSV2iUBUKrPKFUFmh6ebYWHOf1H3P5KaeYhAgLU89uz7ieKU26z6Re47wURWlBuqfE8NL4Hrx1dS+ykiJ57ru9jH1buzmI3en7GSxU8lIUxad6tY3llYk9eX1ST9Liw3lm2R7Gvb2aeRvzcfgwiankpSiKX/RtF8frk3rx2sSepMaG8fQ3u7lx9kafTUmtLsxWFMWv+qfF0a99L1bnllBYasdgMPhkuyp5KYridwaDgXMy4n26TdVsVBQlKKnkpShKUFLJS1GUoKSSl6IoQUklL0VRgpJKXoqiBCWVvBRFCUoqeSmKEpSCYlYJ4DCQq3cQiqI0u3QgubYngiV5KYqi/IpqNiqKEpRU8lIUJSip5KUoSlBSyUtRlKCkkpeiKEFJJS9FUYJSSE1G6L0v5AuACXhLSvmUziE1mhCiPTADaA14gDeklC/oG1XTCCFMwFrggJRytN7xNIUQIg54C+iO9vncKKVcpWtQjSSE+CNwM1o5tgDTpJRV+kZ1ZiFT8/L+MF4BLgW6AtcIIbrqG1WTOIF7pZRdgYHA74K8PAB/AHboHYSPvAAskVKeBfQiSMslhGgL3AX0l1J2RzvwX61vVPUTMskLOBvYLaXcK6V0AHOAMTrH1GhSygIp5Xrv36VoP462+kbVeEKIdsAotNpKUBNCxAIXAG8DSCkdUsoSXYNqGjMQLoQwAxFAvs7x1EsoJa+2wP4aj/MI4h97TUKIDKAP8LPOoTTF88D9gO9v4Nf8MtEuWXtXCLFBCPGWECJS76AaQ0p5AHgG2AcUAMeklF/qG1X9hFLyCklCiCjgY+BuKeVxveNpDCHEaOCQlHKd3rH4iBnoC7wmpewDlAMP6htS4wgh4tFaKJlAKhAphLhO36jqJ5SS1wGgfY3H7bzLgpYQwoKWuN6XUs7XO54mGARcIYTIQWvODxdCzNI3pCbJA/KklCdqwh+hJbNgdCGQLaU8LKWsBuYD5+kcU72EUvJaA3QSQmQKIaxonY4LdY6p0YQQBrQ+lR1Syv/oHU9TSCkfklK2k1JmoH0u30opg+LoXhsp5UFgvxBCeBeNALbrGFJT7AMGCiEivN+5EQTJyYeQSV5SSifwe2Ap2pv/oZRym75RNckg4Hq0WspG77/L9A5KOelO4H0hxGagN/CEvuE0jrf2+BGwHm2YhBF4Q9eg6klNiaMoSlAKmZqXoigti0peiqIEJZW8FEUJSip5KYoSlFTyUhQlKIXUrBJK4BFCuNBOwZ8wx1ezfXgvm1rkvaBYaWFU8lL8rVJK2VvvIJTQo5KXogvvpUIfok1hVAlcK6Xc7a1NvQMkoV38PE1KuU8I0Rr4L9DBu4nb0WY/MAkh3kS7pOUAMEZKWSmEuAu4DW1qoe1SyqCY5kWpP9XnpfhbeI0rBDYKISbVeO6YlLIH8DLarBMALwHvSSl7Au8DL3qXvwh8L6XshXYd4YmrJzoBr0gpuwElwHjv8geBPt7t3Oafoil6UjUvxd9O12ycXeP/57x/nwuM8/49E3ja+/dwYAqAlNIFHPPOiJAtpdzoXWcdkOH9ezPa5TufAJ80sQxKAFI1L0VPnjr+bgh7jb9d/O+APAptZt2+wBrvRHtKCFHJS9HTpBr/n5j//Uf+Nw3xZGCF9+9v0Pq5EEKYvLOZ1koIYQTaSymXAQ8AsUCUb0NX9KaORoq/hQshNtZ4vERKeWLivnjvrAx24BrvsjvRZij9E94Oe+/yPwBvCCFuQqth3Y4282dtTMAsb4IzAC8G+TTNSi3UrBKKLrxnG/tLKY/oHYsSnFSzUVGUoKRqXoqiBCVV81IUJSip5KUoSlBSyUtRlKCkkpeiKEFJJS9FUYLS/wOo8rEaf8Db/gAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "training_vis(train_losses, valid_losses)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.225408, + "end_time": "2021-11-10T09:26:22.625997", + "exception": false, + "start_time": "2021-11-10T09:26:22.400589", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "我们通常会绘制训练/验证曲线来观察模型的拟合情况，上图中我们分别绘制了训练过程中训练集和验证集损失函数变化曲线。可以看到，训练集的损失函数下降很快，但是验证集的损失函数是震荡的，没有明显的下降，这说明模型的学习效果较差，并存在过拟合问题，需要调整相关的参数。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.149048, + "end_time": "2021-11-10T09:26:22.925730", + "exception": false, + "start_time": "2021-11-10T09:26:22.776682", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "#### 模型评估" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.150117, + "end_time": "2021-11-10T09:26:23.225638", + "exception": false, + "start_time": "2021-11-10T09:26:23.075521", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "最后，我们在测试集上评估模型的训练结果。" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:23.525788Z", + "iopub.status.busy": "2021-11-10T09:26:23.525168Z", + "iopub.status.idle": "2021-11-10T09:26:28.319525Z", + "shell.execute_reply": "2021-11-10T09:26:28.319915Z", + "shell.execute_reply.started": "2021-11-10T08:54:41.915357Z" + }, + "papermill": { + "duration": 4.946498, + "end_time": "2021-11-10T09:26:28.320076", + "exception": false, + "start_time": "2021-11-10T09:26:23.373578", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 加载最佳模型权重\n", + "checkpoint = torch.load('../input/ai-earth-model-weights/task03_model_weights.pth')\n", + "model = Model()\n", + "model.load_state_dict(checkpoint['state_dict'])" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:28.625319Z", + "iopub.status.busy": "2021-11-10T09:26:28.624476Z", + "iopub.status.idle": "2021-11-10T09:26:28.626951Z", + "shell.execute_reply": "2021-11-10T09:26:28.626515Z", + "shell.execute_reply.started": "2021-11-10T08:54:49.04339Z" + }, + "papermill": { + "duration": 0.156625, + "end_time": "2021-11-10T09:26:28.627069", + "exception": false, + "start_time": "2021-11-10T09:26:28.470444", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# 测试集路径\n", + "test_path = '../input/ai-earth-tests/'\n", + "# 测试集标签路径\n", + "test_label_path = '../input/ai-earth-tests-labels/'" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:28.929724Z", + "iopub.status.busy": "2021-11-10T09:26:28.929173Z", + "iopub.status.idle": "2021-11-10T09:26:31.894697Z", + "shell.execute_reply": "2021-11-10T09:26:31.895219Z", + "shell.execute_reply.started": "2021-11-10T08:54:56.246479Z" + }, + "papermill": { + "duration": 3.119171, + "end_time": "2021-11-10T09:26:31.895385", + "exception": false, + "start_time": "2021-11-10T09:26:28.776214", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "import os\n", + "\n", + "# 读取测试数据和测试数据的标签，并记录每个测试样本的起始月份用于之后构造月份特征\n", + "files = os.listdir(test_path)\n", + "X_test = []\n", + "y_test = []\n", + "first_months = [] # 样本起始月份\n", + "for file in files:\n", + " X_test.append(np.load(test_path + file))\n", + " y_test.append(np.load(test_label_path + file))\n", + " first_months.append(int(file.split('_')[2]))" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:32.211381Z", + "iopub.status.busy": "2021-11-10T09:26:32.210614Z", + "iopub.status.idle": "2021-11-10T09:26:32.248569Z", + "shell.execute_reply": "2021-11-10T09:26:32.249226Z", + "shell.execute_reply.started": "2021-11-10T08:55:00.037468Z" + }, + "papermill": { + "duration": 0.192811, + "end_time": "2021-11-10T09:26:32.249387", + "exception": false, + "start_time": "2021-11-10T09:26:32.056576", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "((103, 12, 24, 72, 4), (103, 24))" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "X_test = np.array(X_test)\n", + "y_test = np.array(y_test)\n", + "X_test.shape, y_test.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:32.563311Z", + "iopub.status.busy": "2021-11-10T09:26:32.562190Z", + "iopub.status.idle": "2021-11-10T09:26:35.298239Z", + "shell.execute_reply": "2021-11-10T09:26:35.297765Z", + "shell.execute_reply.started": "2021-11-10T08:55:43.648698Z" + }, + "papermill": { + "duration": 2.896704, + "end_time": "2021-11-10T09:26:35.298373", + "exception": false, + "start_time": "2021-11-10T09:26:32.401669", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(103, 12, 24, 72, 1)" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造一个维度为103*12*24*72的矩阵，矩阵中的每个值为所在月份的sin函数值\n", + "test_month_sin = np.zeros((103, 12, 24, 72, 1))\n", + "for y in range(103):\n", + " for m in range(12):\n", + " for lat in range(24):\n", + " for lon in range(72):\n", + " test_month_sin[y, m, lat, lon] = math.sin(2 * math.pi * ((m + first_months[y]-1) % 12) / 12)\n", + " \n", + "test_month_sin.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:35.604627Z", + "iopub.status.busy": "2021-11-10T09:26:35.603568Z", + "iopub.status.idle": "2021-11-10T09:26:38.163083Z", + "shell.execute_reply": "2021-11-10T09:26:38.162135Z", + "shell.execute_reply.started": "2021-11-10T08:55:46.843147Z" + }, + "papermill": { + "duration": 2.71433, + "end_time": "2021-11-10T09:26:38.163235", + "exception": false, + "start_time": "2021-11-10T09:26:35.448905", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(103, 12, 24, 72, 1)" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造一个维度为103*12*24*72的矩阵，矩阵中的每个值为所在月份的cos函数值\n", + "test_month_cos = np.zeros((103, 12, 24, 72, 1))\n", + "for y in range(103):\n", + " for m in range(12):\n", + " for lat in range(24):\n", + " for lon in range(72):\n", + " test_month_cos[y, m, lat, lon] = math.cos(2 * math.pi * ((m + first_months[y]-1) % 12) / 12)\n", + " \n", + "test_month_cos.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:38.480942Z", + "iopub.status.busy": "2021-11-10T09:26:38.477524Z", + "iopub.status.idle": "2021-11-10T09:26:38.601413Z", + "shell.execute_reply": "2021-11-10T09:26:38.600796Z", + "shell.execute_reply.started": "2021-11-10T08:55:49.806402Z" + }, + "papermill": { + "duration": 0.283781, + "end_time": "2021-11-10T09:26:38.601551", + "exception": false, + "start_time": "2021-11-10T09:26:38.317770", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(103, 12, 24, 72, 6)" + ] + }, + "execution_count": 38, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 构造测试集\n", + "X_test = np.concatenate([X_test, test_month_sin, test_month_cos], axis=-1)\n", + "X_test.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:38.913539Z", + "iopub.status.busy": "2021-11-10T09:26:38.912509Z", + "iopub.status.idle": "2021-11-10T09:26:38.949669Z", + "shell.execute_reply": "2021-11-10T09:26:38.949186Z", + "shell.execute_reply.started": "2021-11-10T08:56:29.728334Z" + }, + "papermill": { + "duration": 0.195356, + "end_time": "2021-11-10T09:26:38.949790", + "exception": false, + "start_time": "2021-11-10T09:26:38.754434", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "testset = AIEarthDataset(X_test, y_test)\n", + "testloader = DataLoader(testset, batch_size=batch_size, shuffle=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "metadata": { + "execution": { + "iopub.execute_input": "2021-11-10T09:26:39.264750Z", + "iopub.status.busy": "2021-11-10T09:26:39.263435Z", + "iopub.status.idle": "2021-11-10T09:26:39.402020Z", + "shell.execute_reply": "2021-11-10T09:26:39.401284Z", + "shell.execute_reply.started": "2021-11-10T08:56:42.108524Z" + }, + "papermill": { + "duration": 0.298967, + "end_time": "2021-11-10T09:26:39.402188", + "exception": false, + "start_time": "2021-11-10T09:26:39.103221", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "4it [00:00, 65.03it/s]" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Score: 14.946\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "# 在测试集上评估模型效果\n", + "model.eval()\n", + "model.to(device)\n", + "preds = np.zeros((len(y_test),24))\n", + "for i, data in tqdm(enumerate(testloader)):\n", + " data, labels = data\n", + " data = data.to(device)\n", + " labels = labels.to(device)\n", + " pred = model(data)\n", + " preds[i*batch_size:(i+1)*batch_size] = pred.detach().cpu().numpy()\n", + "s = score(y_test, preds)\n", + "print('Score: {:.3f}'.format(s))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.15343, + "end_time": "2021-11-10T09:26:39.715114", + "exception": false, + "start_time": "2021-11-10T09:26:39.561684", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## 总结" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.153001, + "end_time": "2021-11-10T09:26:40.021240", + "exception": false, + "start_time": "2021-11-10T09:26:39.868239", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "- 该方案在数据处理部分采用了滑窗来构造数据集，这是序列预测问题中常用的增加数据量的方法。另外，该方案中增加了一组月份特征，个人认为在时序场景中增加的这组特征收益不高，更多的是通过模型挖掘序列中的依赖关系，并且由于维度增加会使得训练数据占用的资源大大增加，对模型的效果提升不明显。不过在其他场景中这种特征构造方法仍然是值得借鉴的。\n", + "- 该方案没有选择时空序列预测领域的现有模型，而是选择自己设计模型，方案中的这种构造模型的思路非常适合初学者学习，灵活地将不同模型串行或并行组合能够结合模型各自的优势，这种模型构造方法需要注意的是一个模型的输出维度与另一个模型接受的输入维度要相互匹配。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 作业\n", + "\n", + "学有余力的同学可以尝试在不增加月份特征的情况下使用CNN+LSTM模型进行预测，同时尝试修改模型参数或层数比较模型在测试集上的评分。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "papermill": { + "duration": 0.154401, + "end_time": "2021-11-10T09:26:40.329808", + "exception": false, + "start_time": "2021-11-10T09:26:40.175407", + "status": "completed" + }, + "tags": [] + }, + "source": [ + "## 参考文献\n", + "1. “学习AI的打工人”经验分享：https://tianchi.aliyun.com/notebook-ai/detail?spm=5176.12586969.1002.18.561d5330HKwYOW&postId=196536" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + }, + "papermill": { + "default_parameters": {}, + "duration": 711.969076, + "end_time": "2021-11-10T09:26:42.429334", + "environment_variables": {}, + "exception": null, + "input_path": "__notebook__.ipynb", + "output_path": "__notebook__.ipynb", + "parameters": {}, + "start_time": "2021-11-10T09:14:50.460258", + "version": "2.3.3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}