diff --git a/CH5-集成学习之blending与stacking/1.png b/CH5-集成学习之blending与stacking/1.png new file mode 100644 index 0000000..878a94c Binary files /dev/null and b/CH5-集成学习之blending与stacking/1.png differ diff --git a/CH5-集成学习之blending与stacking/2.png b/CH5-集成学习之blending与stacking/2.png new file mode 100644 index 0000000..6af3e01 Binary files /dev/null and b/CH5-集成学习之blending与stacking/2.png differ diff --git a/CH5-集成学习之blending与stacking/3.jpg b/CH5-集成学习之blending与stacking/3.jpg new file mode 100644 index 0000000..4110a17 Binary files /dev/null and b/CH5-集成学习之blending与stacking/3.jpg differ diff --git a/CH5-集成学习之blending与stacking/4.jpg b/CH5-集成学习之blending与stacking/4.jpg new file mode 100644 index 0000000..7eed211 Binary files /dev/null and b/CH5-集成学习之blending与stacking/4.jpg differ diff --git a/CH5-集成学习之blending与stacking/5.png b/CH5-集成学习之blending与stacking/5.png new file mode 100644 index 0000000..2242b0e Binary files /dev/null and b/CH5-集成学习之blending与stacking/5.png differ diff --git a/CH5-集成学习之blending与stacking/Stacking.ipynb b/CH5-集成学习之blending与stacking/Stacking.ipynb new file mode 100644 index 0000000..eff9c3b --- /dev/null +++ b/CH5-集成学习之blending与stacking/Stacking.ipynb @@ -0,0 +1,690 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 1. 导言" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在前几个章节中,我们学习了关于回归和分类的算法,同时也讨论了如何将这些方法集成为强大的算法的集成学习方式,分别是Bagging和Boosting。本章我们继续讨论集成学习方法的最后一个成员--Stacking,这个集成方法在比赛中被称为“懒人”算法,因为它不需要花费过多时间的调参就可以得到一个效果不错的算法,同时,这种算法也比前两种算法容易理解的多,因为这种集成学习的方式不需要理解太多的理论,只需要在实际中加以运用即可。 stacking严格来说并不是一种算法,而是精美而又复杂的,对模型集成的一种策略。Stacking集成算法可以理解为一个两层的集成,第一层含有多个基础分类器,把预测的结果(元特征)提供给第二层, 而第二层的分类器通常是逻辑回归,他把一层分类器的结果当做特征做拟合输出预测结果。在介绍Stacking之前,我们先来对简化版的Stacking进行讨论,也叫做Blending,接着我们对Stacking进行更深入的讨论。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 2. Blending集成学习算法" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "不知道大家小时候有没有过这种经历:老师上课提问到你,那时候你因为开小差而无法立刻得知问题的答案。就在你彷徨的时候,由于你平时人缘比较好,因此周围的同学向你伸出援手告诉了你他们脑中的正确答案,因此你对他们的答案加以总结和分析最终的得出正确答案。相信大家都有过这样的经历,说这个故事的目的是为了引出集成学习家族中的Blending方式,这种集成方式跟我们的故事是十分相像的。如图:(图片来源:https://blog.csdn.net/maqunfi/article/details/82220115) \n", + "\n", + "![jupyter](./1.png) \n", + "下面我们来详细讨论下这个Blending集成学习方式: \n", + " - (1) 将数据划分为训练集和测试集(test_set),其中训练集需要再次划分为训练集(train_set)和验证集(val_set);\n", + " - (2) 创建第一层的多个模型,这些模型可以使同质的也可以是异质的;\n", + " - (3) 使用train_set训练步骤2中的多个模型,然后用训练好的模型预测val_set和test_set得到val_predict, test_predict1;\n", + " - (4) 创建第二层的模型,使用val_predict作为训练集训练第二层的模型;\n", + " - (5) 使用第二层训练好的模型对第二层测试集test_predict1进行预测,该结果为整个测试集的结果。 \n", + " \n", + "![jupyter](./2.png) \n", + "(图片来源:https://blog.csdn.net/sinat_35821976/article/details/83622594) \n", + "\n", + "在这里,笔者先来梳理下这个过程: \n", + "在(1)步中,总的数据集被分成训练集和测试集,如80%训练集和20%测试集,然后在这80%的训练集中再拆分训练集70%和验证集30%,因此拆分后的数据集由三部分组成:训练集80%* 70%\n", + "、测试集20%、验证集80%* 30% 。训练集是为了训练模型,测试集是为了调整模型(调参),测试集则是为了检验模型的优度。 \n", + "在(2)-(3)步中,我们使用训练集创建了K个模型,如SVM、random forests、XGBoost等,这个是第一层的模型。 训练好模型后将**验证集**输入模型进行预测,得到K组不同的输出,我们记作$A_1,...,A_K$,然后将测试集输入K个模型也得到K组输出,我们记作$B_1,...,B_K$,其中$A_i$的样本数与验证集一致,$B_i$的样本数与测试集一致。如果总的样本数有10000个样本,那么使用5600个样本训练了K个模型,输入验证集2400个样本得到K组2400个样本的结果$A_1,...,A_K$,输入测试集2000个得到K组2000个样本的结果$B_1,...,B_K$ 。 \n", + "在(4)步中,我们使用K组2400个样本的验证集结果$A_1,...,A_K$作为第二层分类器的特征,验证集的2400个标签为因变量,训练第二层分类器,得到2400个样本的输出。 \n", + "在(5)步中,将输入测试集2000个得到K组2000个样本的结果$B_1,...,B_K$放入第二层分类器,得到2000个测试集的预测结果。 \n", + "\n", + "![jupyter](./3.jpg)\n", + "\n", + "以上是Blending集成方式的过程,接下来我们来分析这个集成方式的优劣: \n", + "其中一个最重要的优点就是实现简单粗暴,没有太多的理论的分析。但是这个方法的缺点也是显然的:blending只使用了一部分数据集作为留出集进行验证,也就是只能用上数据中的一部分,实际上这对数据来说是很奢侈浪费的。 \n", + "关于这个缺点,我们以后再做改进,我们先来用一些案例来使用这个集成方式。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "# 加载相关工具包\n", + "import numpy as np\n", + "import pandas as pd \n", + "import matplotlib.pyplot as plt\n", + "plt.style.use(\"ggplot\")\n", + "%matplotlib inline\n", + "import seaborn as sns" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The shape of training X: (5600, 2)\n", + "The shape of training y: (5600,)\n", + "The shape of test X: (2000, 2)\n", + "The shape of test y: (2000,)\n", + "The shape of validation X: (2400, 2)\n", + "The shape of validation y: (2400,)\n" + ] + } + ], + "source": [ + "# 创建数据\n", + "from sklearn import datasets \n", + "from sklearn.datasets import make_blobs\n", + "from sklearn.model_selection import train_test_split\n", + "data, target = make_blobs(n_samples=10000, centers=2, random_state=1, cluster_std=1.0 )\n", + "## 创建训练集和测试集\n", + "X_train1,X_test,y_train1,y_test = train_test_split(data, target, test_size=0.2, random_state=1)\n", + "## 创建训练集和验证集\n", + "X_train,X_val,y_train,y_val = train_test_split(X_train1, y_train1, test_size=0.3, random_state=1)\n", + "print(\"The shape of training X:\",X_train.shape)\n", + "print(\"The shape of training y:\",y_train.shape)\n", + "print(\"The shape of test X:\",X_test.shape)\n", + "print(\"The shape of test y:\",y_test.shape)\n", + "print(\"The shape of validation X:\",X_val.shape)\n", + "print(\"The shape of validation y:\",y_val.shape)" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": {}, + "outputs": [], + "source": [ + "# 设置第一层分类器\n", + "from sklearn.svm import SVC\n", + "from sklearn.ensemble import RandomForestClassifier\n", + "from sklearn.neighbors import KNeighborsClassifier\n", + "\n", + "clfs = [SVC(probability = True),RandomForestClassifier(n_estimators=5, n_jobs=-1, criterion='gini'),KNeighborsClassifier()]\n", + "\n", + "# 设置第二层分类器\n", + "from sklearn.linear_model import LinearRegression\n", + "lr = LinearRegression()\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [], + "source": [ + "# 输出第一层的验证集结果与测试集结果\n", + "val_features = np.zeros((X_val.shape[0],len(clfs))) # 初始化验证集结果\n", + "test_features = np.zeros((X_test.shape[0],len(clfs))) # 初始化测试集结果\n", + "\n", + "for i,clf in enumerate(clfs):\n", + " clf.fit(X_train,y_train)\n", + " val_feature = clf.predict_proba(X_val)[:, 1]\n", + " test_feature = clf.predict_proba(X_test)[:,1]\n", + " val_features[:,i] = val_feature\n", + " test_features[:,i] = test_feature\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([1., 1., 1., 1., 1.])" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 将第一层的验证集的结果输入第二层训练第二层分类器\n", + "lr.fit(val_features,y_val)\n", + "# 输出预测的结果\n", + "from sklearn.model_selection import cross_val_score\n", + "cross_val_score(lr,test_features,y_test,cv=5)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "可以看到,在每一折的交叉验证的效果都是非常好的,这个集成学习方法在这个数据集上是十分有效的,不过这个数据集是我们虚拟的,因此大家可以把他用在实际数据上看看效果。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**作业: \n", + "留个小作业吧,我们刚刚的例子是针对人造数据集,表现可能会比较好一点,因为我们使用Blending方式对iris数据集进行预测,并用第四章的决策边界画出来,找找规律。**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 3. Stacking集成学习算法" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "基于前面对Blending集成学习算法的讨论,我们知道:Blending在集成的过程中只会用到验证集的数据,对数据实际上是一个很大的浪费。为了解决这个问题,我们详细分析下Blending到底哪里出现问题并如何改进。在Blending中,我们产生验证集的方式是使用分割的方式,产生一组训练集和一组验证集,这让我们联想到交叉验证的方式。顺着这个思路,我们对Stacking进行建模(如下图): \n", + "\n", + "![jupyter](./4.jpg) \n", + "\n", + " - 首先将所有数据集生成测试集和训练集(假如训练集为10000,测试集为2500行),那么上层会进行5折交叉检验,使用训练集中的8000条作为训练集,剩余2000行作为验证集(橙色)。 \n", + " - 每次验证相当于使用了蓝色的8000条数据训练出一个模型,使用模型对验证集进行验证得到2000条数据,并对测试集进行预测,得到2500条数据,这样经过5次交叉检验,可以得到中间的橙色的5* 2000条验证集的结果(相当于每条数据的预测结果),5* 2500条测试集的预测结果。     \n", + " - 接下来会将验证集的5* 2000条预测结果拼接成10000行长的矩阵,标记为$A_1$,而对于5* 2500行的测试集的预测结果进行加权平均,得到一个2500一列的矩阵,标记为$B_1$。 \n", + " - 上面得到一个基模型在数据集上的预测结果$A_1$、$B_1$,这样当我们对3个基模型进行集成的话,相于得到了$A_1$、$A_2$、$A_3$、$B_1$、$B_2$、$B_3$六个矩阵。 \n", + " - 之后我们会将$A_1$、$A_2$、$A_3$并列在一起成10000行3列的矩阵作为training data,$B_1$、$B_2$、$B_3$合并在一起成2500行3列的矩阵作为testing  data,让下层学习器基于这样的数据进行再训练。 \n", + " - 再训练是基于每个基础模型的预测结果作为特征(三个特征),次学习器会学习训练如果往这样的基学习的预测结果上赋予权重w,来使得最后的预测最为准确。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "下面,我们来实际应用下Stacking是如何集成算法的:(参考案例:https://www.cnblogs.com/Christina-Notebook/p/10063146.html) \n", + "![jupyter](./5.png) \n", + "**由于sklearn并没有直接对Stacking的方法,因此我们需要下载mlxtend工具包(pip install mlxtend)**" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3-fold cross validation:\n", + "\n", + "Accuracy: 0.91 (+/- 0.01) [KNN]\n", + "Accuracy: 0.95 (+/- 0.01) [Random Forest]\n", + "Accuracy: 0.91 (+/- 0.02) [Naive Bayes]\n", + "Accuracy: 0.93 (+/- 0.02) [StackingClassifier]\n" + ] + } + ], + "source": [ + "# 1. 简单堆叠3折CV分类\n", + "from sklearn import datasets\n", + "\n", + "iris = datasets.load_iris()\n", + "X, y = iris.data[:, 1:3], iris.target\n", + "from sklearn.model_selection import cross_val_score\n", + "from sklearn.linear_model import LogisticRegression\n", + "from sklearn.neighbors import KNeighborsClassifier\n", + "from sklearn.naive_bayes import GaussianNB \n", + "from sklearn.ensemble import RandomForestClassifier\n", + "from mlxtend.classifier import StackingCVClassifier\n", + "\n", + "RANDOM_SEED = 42\n", + "\n", + "clf1 = KNeighborsClassifier(n_neighbors=1)\n", + "clf2 = RandomForestClassifier(random_state=RANDOM_SEED)\n", + "clf3 = GaussianNB()\n", + "lr = LogisticRegression()\n", + "\n", + "# Starting from v0.16.0, StackingCVRegressor supports\n", + "# `random_state` to get deterministic result.\n", + "sclf = StackingCVClassifier(classifiers=[clf1, clf2, clf3], # 第一层分类器\n", + " meta_classifier=lr, # 第二层分类器\n", + " random_state=RANDOM_SEED)\n", + "\n", + "print('3-fold cross validation:\\n')\n", + "\n", + "for clf, label in zip([clf1, clf2, clf3, sclf], ['KNN', 'Random Forest', 'Naive Bayes','StackingClassifier']):\n", + " scores = cross_val_score(clf, X, y, cv=3, scoring='accuracy')\n", + " print(\"Accuracy: %0.2f (+/- %0.2f) [%s]\" % (scores.mean(), scores.std(), label))" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkkAAAHjCAYAAAA374shAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd3hc1Z3/8feZKmnUJfdeMbYBN4oFAQOmmd4m9ECKk02WNJJsQsqSX3o2ZEmySXZZCCw9lxowBkKzDRhwx9jYuMhykS3balYbTb2/P0aSNZo70oxmRtO+r+fRA7pz59xzpNHH55577rlK13WEEEIIIUQoU6orIIQQQgiRjqSTJIQQQghhQDpJQgghhBAGpJMkhBBCCGFAOklCCCGEEAakkySEEEIIYUA6SUIIITKOUuphpdQbqa6HyG7SSRJhjMJHKTVPKVWnlHpeKXWJUkpXSh1QShX0916l1D1d+z5vcJwapdSPktcSIUQydP2d611f/q4seEQpNSbVdRsKSqnbe7W/99d/p0Hdzuqqy8RU1yUbSCdJDEgpdSGwEngOuBZwdb1UDnwviiI6gSuVUouSUkEhRCq8A4wCxgM3AXOBp1Nao6HlJ9j+3l/R5KEhpZRVKaUSVDeRINJJEv1SSt0KvAT8Rtf1r+q6Huj18u+B70Zx9lgLaMDvlVLymRMiO3h0Xa/Tdb1W1/VVwP3AQqVUcfcOSqkLlFIrlFKNSqljSqmVSqnTehfSNerxVaXUo0qpVqXUfqXU9/rsU6aU+rtSql0pdVgp9XNA9dnHqpT6tVKqVinlUUp9opS6yeBYd/Yqa59S6jqlVIlS6vGu41crpa6N5gfQ1f7eXy29jrVEKbVeKeVWSh1RSv1FKeXo9frDSqk3uupTA7gBR9drdyqltiulOpVSO5VSP1RKWXq990ql1EalVIdSqlkptUYpNbdr9Oidrt32dLV3RTRtEcbkHywRUVdQPQh8Wdf1nxvs8r/AHuCXURT3b8CJwOcSV0MhRDpQSo0GriM4uuLv9VIh8GfgDKAK2Am8qpSq6FPEvwOrgDnAfwC/UUqd2+v1vwHzgcuB84CJwNV9yvgl8CXgm8Bs4DHgMaXU+X32+yGwHDgFWAY8AjwFvE5wNOxl4BGDOkZNKXUy8GKvNn0OuAzoeznutK72XNVVn06l1D3Ad4AfEMzMbwBfJvgzQik1kuCI3ZPALGAhcB/gA/YDV/YqexRwzWDbIQBd1+VLvkK+gIcJntXowK0Gry/qem0scBEQAOb3eu8bvfa9B9jV9f+/Ag4Cjq7va4Afpbq98iVf8hXbV9ffuQ9oAzq68kAHfjfA+0xAE3Bzr2068Mc++20HftX1/1O79rmg1+s2giPUb3R9X9CVWV/tU87zwFt9jnVfr++HdW37U69tZV3bLuunHbd37dPW52t61+uPAmv6vOfKrqyc0Otn2AwU9tqnoOvneXGf994GNHf9/9yuY0+MULez+ntdvmL7kpEkEcl2YBvwg66zREO6rr8GvEbw0ttAfgmYCY4qCSEy24cER0lOA34GfAD8uPcOSqlJXZfRdimlWoAWoASY0KesTX2+rwVGdP3/zK7/ru5+Udd1D7C21/5TCXacVvUpZyXB0ZbePupVzlGCI1+be21rAjzAcPrnJ9j+3l97ul6bFaEuqld7ALbput7W6/tZQD7wrFKqrfsL+B+gRCk1rKuurwFbum6k+YZSatwAdRWDJJ0kEclR4GyCk65XKaX6hlpvdwFVA13H13W9lWCIfkcpNTZhNRVCpIJL1/Vduq5v0XX9J8BegpfWeltGcGL31whecpsDHCHYoenN0+d7neP/PsUymVnv870y2OY1eF/fbb2PH/lgwfb3/updTt/jGm1v7/Na9zGvJ7TzdRIwDWjUdd0PXELwMt1agjfT7FBKXTZQfUXspJMkItJ1vZ7gH+JR4B2l1LQI+31CcH7SbwgPv74eBHYRvPQmhMge9wCfU0otAOia0zMT+LWu66915UQnA4/Q9LW1679V3RuUUjbg1F777CJ4ue2cPu89u9f7h9JWg7qcQ7CD9MkA7+sEJht0wHZ1dZDQg9bouv5LXdfPJjhKdUdXGd0dTnPCWpPDpJMk+qXrejNwAcEQWqWU6jt03e0nQCXBCYj9lecnOPJ0MzAygVUVQqSQruvbCY4cdZ8ANRE8wfqSUmq6UmohwcnGrghFRCp3F8FJ0H9WSp2rlJoJPAAU9dqnA/gj8DOl1PVKqWlKqbsJzgOK5saSRPsPYJ5S6vdKqRlKqYuBPwGP67q+L9Kbui69/RL4pVLqX5VSJyilZimlblBK/QZAKVWllPqxUup0pdT4ronpJ3O887WX4NynJUqp4UqpkmQ2NNtJJ0kMqOsPdwmwAVgBzDPYpx74OcHr6QOV9zrBu0vsCa2oECLVfgssVkqdrweXC7kemEJwHs3DBO/COjSIcj9PcN7SMoKjJrUEJ2X39kOCI9r3ERyRuQW4Rdf1NwdxvLjour4ZuILg6NFHBCdyvwx8JYr3/gz4FvDFrve+2/V9Tdcuxwje0fYPgncL/g14nOC8MHRdP0zwzrjvE/xZ/yMxrcpNStcjXTYVQgghhMhdMpIkhBBCCGFAOklCCCGEEAakkySEEEIIYUA6SUIIIYQQBqSTJIQQQghhwDLwLrF7dNMDekdHRzKKThsFBQVkextB2plNktnGL1d9PZZVkdNaLuQXyGc+m0g74zOpeCoXzl5imGFJGUlSKmvyMqJcaCNIO7NJLrQxEXLl55QL7cyFNoK0M5nkcpsQQgghhAHpJAkhhBBCGJBOkhBCCCGEgaRM3BZCDI7DVIjNlJxH2uWpPGyWvLjK8ATctAfaElQjIUQ2SWZ+QWoyTDpJQqQJh6mQQlsRJGluotVqI+APxFWGTbeBB+koCSFCJDu/IDUZJpfbhEgTNpM9qQGTEIqknikKITJTRuQXxJxh0kkSQgghhDAgl9uEED02vLeJB3/3EAF/gMVXn8+1d1yV6ioJIUTUEp1hMpIkhADA7w9w/28e5Md/ups/PvufvPvqe+yvPpDqagkhRFSSkWEykiREBnrmyRUcPeYP2z6sxMx1Ny4aVJk7t+xi1NiRjBw7AoCzLqpizYq1jJs8Np6qCiFEiGTkFyQnw6STJEQGOnrMT8fkJeHbq5cPuszGo41Ujqzo+b5ieAU7tuwcdHlCCGEkGfkFyckwudwmhABA1/WwbbnyTCghROZLRoZJJ0kIAQTPuurrGnq+bzjSQPmwshTWSAghopeMDJNOkhACgGmzpnBo/yEO1x7B6/Xx7murOfWcBamulhBCRCUZGSZzkoQQAJgtZr70b5/np1/7BYFAgPOvOJfxU8alulpCCBGVZGSYdJKEyEDDSsyGkxyHlZjjKnf+WfOYf9a8uMoQQoj+JCu/IPEZJp0kITJQPLfJCiFEKmVSfg3YSXI6nScAf++1aTLwE03T7ktarYQQIgEkv4QQ8Riwk6Rp2qfAHACn02kGaoHnk1wvIYSIm+SXECIesd7ddj6wW9O0vcmojBBCJJHklxAiJrHOSboBeNLoBafTuRRYCjDrhqlMmTIlzqqlN7PZhMPhSGkdmo40sfL1d+jwtlFgLeScCz5D2fDErmuTDu0cCunQzjyVh9VqS1r5ZpPCnpcXdzkmswmHnpGfCcmvXlL9mZf8Spx0aGey8wuSl2ElxcUR91VGK1QacTqdNuAgMEvTtMP97fvYRw/q7e3tUZWbqRwOB6lsY/PRZl548QUqzinEbDPj9/hpWNnGVVdcRemw0oQdJ9XtHCrp0M4ySwW2JIaMPS8Pd2dn3OV4vB6afA0h275c9fW0Xppb8itcKj/zkl+JlQ7tTHZ+QfIybFLxVC6cvcQww2K53HYJsGGggBFDY/WK93sCBsBsM1NxTiGrV7yf4pqJTPane/7C587/Il+//q5UVyXRJL/SiOSXSIZk5FcsnaQbiTBULYaey9fREzDdzDYzLp8rRTUS2eC8yxfxk/+6O9XVSAbJrzQi+SWSIRn5FVUnyel0FgAXAM8l9Ohi0PItBfg9/pBtfo+ffEt+imokUkHXdd7++0uGD3YcjFnzZ1JUUpiQstKF5Ff6kfwSkBn5FdXEbU3TOoCKhB5ZxKVq0cII1/QXG+7ffLSZ1Svex+XrIN9SQNWihQm99p8ucqWd3XZt2ELZrnfZvXEiU+edlOrqpCXJr/QTa35Bbvxt50Ibe8uE/JIH3Gao0mGlXHXFVZg3OnC9p2Pe6Ig46bF7kqR/bjv5Z5rwz23nhRdfoPlocwpqnjy50s5uuq5T995bfL3KwaF330rY2ZgQyRZLfkFu/G3nQht7y5T8kseSZLDSYaUsuf6SAffrb5JkpPd3n9F48WDFlhFnNINpZybbtWELi8saUMrK+aUNfLJxS9qejQnRV7T5BbH/bUt+pb9MyS8ZScoBsU6S7H1GY19IxpzR5NJk0O6zsNPHBs9zzhhnSeuzMSHiEcvftuRX+suk/JJOUg6IdZJkpt6em0uTQY+fhQWX9lBKcX5pA7s3bomr3Ht/cB/fv/1HHNx7kC9e/BXeeOGtRFRXiLjE8rct+ZX+Mim/5HJbDoh1kqTL10F+Bp7RDGYyaKY6tHMXzR0jeG/H8fXPdF2nc+euuIas7/rVNxNRPSESKpa/bcmv9JdJ+SWdpBzQPUkyeNeEi3yLg6uuWBzxGn3wjKY9ZOg3eEaT3sv7x9rOTHb2DVenugpCDJlY/rYlv9JfJuWXdJJyRCyTJHuf0ZjyTBl6RpN+17aFEIMXbYZJfolEkjlJIkzv23Pd76sBb89NF7l2C60QIpzkl0gkGUkShrrP2tLhwYnRyrVbaIUQxiS/RKJIJynHZdMKr5k6YVMIMTiSXyLZ5HJbDsu24d1cuoVWiFwn+SWGgnSSclimricSSdWihTSsbOsJmu4Jm1WLFqa4Zpmjvq6eHy/9Kf96zbf4+nXf5qUnlqe6SkIYkvwSfSUjv+RyWw7LtuHdXLqFNllMZjO3f+tWppw4GVe7i7tu/j5zzjiZcZPHprpqQoSQ/BJ9JSO/pJOUwzJ1PZH+xLLUQaZrONLIG2++Qbu3DYe1kMXnL6ZieHlcZZYPK6N8WBkA+Y58xk4aQ8ORRukkibQj+ZXZMiW/pJOUI4wmOPa3wmu6PCAymyZmJlLDkUYeefYhys92YLOZ6fQEv7/t2jviDppuRw4eYc+ne5g+e2pCyhMiHn2zYNZJs3h75VtpnV9G9ZYMy6z8kjlJOSDSBEegZz0R13t6z3oiQFo8IDLbJmYm0htvvkH52Y6Q+RjlZzt44803ElK+q6OT33znXj5/1+0UFBYkpEwhBssoC95+7y3OPfO8tM2vSPWWDMus/IpqJMnpdJYCDwCzCS4F+nlN0zJzdlwOGmj9jb7Du8uffiWm9ToinSnFewYl64ZE1u5tw2YwH6Pd2xZ32T6vj99+517OXvIZFp5/etzlpZrkV+aLlAVbN26NO79AMmyoZVJ+RTuS9AfgVU3TZgCnANviPrIYMi5fR8h1e+h/gmMs+0c6U9q7fV/cZ1Cx1juXOKyFhrcLO6yFcZWr6zp//n//zdhJY7jylsviKiuNSH5luFiyINbckAwbepmUXwN2kpxOZzFwNvAggKZpHk3TcnusMMPEuv5GLPtHOlN68Zllcd+eK+uGRLb4/MU0rmoPuV24cVU7i8+P7/lU2zZ9yoqXV/Hx2i1864bv8q0bvsv6dzckosopIfmVHWLJglhzQzJs6GVSfkVzuW0ycBR4yOl0ngKsB76haVrIWu9Op3MpsBRg1g1TmTJlSlwVS3dmswmHIzPuoli85HyeevrvVHRdA/Z7/DS96+KG668wbMPiJefz+BOPY5sLygq6Fzwb4eabwvf34iHQ4WffqkP49QBmZWL0guH4LV6sedaQfU15Jtx4o/65xVrveKTD7zNP5WG12qLad/T40Xzp5i/z6qv/pN3bisNaxLU3X0jF8MqI7zGbFPa8vH7LnbtwDss/ebHffUxmEw49Mz77SH5FlA6f+WjFkgWx5BdkR4alw+8y2fkFA2dYNPkF4RlWUlwccd9oOkkWYB5wp6ZpHzqdzj8A3wd+3HsnTdPuB+4HeOyjB/VMeV7OYGXSM4FsDhtXXHo5q1e8T7vPRb4lnysuPQ+bw2bYho6ODrydXlo3uoJjjQHI68yno6MDW3voH4GvPUDNqn2MOK8cs82E3xOg+q19qDY73k5v2O25VqL/ucVa73ikw+/TZskj4A9EvX9hcSHXOa8J2ebu7Iy4vz0vr9/Xo+Xxemj3ZcZnH8mviNLhMx+tWLIglvyC7MiwdPhdJju/IHkZdszcEnHfaDpJB4ADmqZ92PX9MwRDRgyRRNxCGsv6G6tXvE/eiRYOr2rrORMrPbvIcLKhMikqFpRgtioAzNbg9/aPSmhY2WZ4e26y6i2EAcmvFEvULfDRZkEs+QWSYaJ/A85J0jStDtjvdDpP6Np0PvBJUmsleqTiFtK6Q3Xsfb+W8ddXMvGG4Yy/vpK979dSd6gubF/d6g+ua9FhItCmQ4eJiuHl2IqthssL5Pr6IGJoSX6lVrrnF0iGif5Fu5jkncDjTqfTBlQDdySvSqK3VNxCWltTx/g7hmPOC/ahzXkmxl81nH0PhYdMvqUAf6CdkrKSnm3dExPlDEqkCcmvFEn3/ALJMNG/qDpJmqZtAhYkuS7CQCqeT5RfbsNkDR1kNFlN5Jfbw/atWrSQZ559hrz5pp6h7c71Aa67djF7t+/jxWeW4TW7sfrtXHHdZUyYMT5p9c50noAbm24Dleqa9EMP1jOTSH6lTrrnF0iGJUpG5BfEnGHyWJI0l4rnE9nJBx8EP+168L8+HTvGdxUEvAEaN7WD0kFX5HnzObC7lmVvvsTYqysx2/Pxu/08+ewT3HjtTRIyEbQH2sADNpNxmMfLZDbh8XriKsMTcAfrKUQUMiG/QDIsEZKdX5CaDFO6rsd1QCO5cHfIUN1N0H1NP3zyYHD5/WgnREY6IzKaVHmsoYUnn32CsVdVYrGb8bn9HHihnhuvvYmSiuKQ/V3tndjO8oeF4Ka/7GTSF4Zhtvfa7vbT9LyHW798U9o9yygd7g5JtmS28ctVX0/388eo5UJ+wdB85hOVX2CcYX3zKNb8qlq0kNUr3sc/N7wjFynDjmodnDB7muRXCiSrnZOKp3Lh7CWGGSbPbktzpcNKB3y+2kATIvdu38eTzz5B2dU2Rl1fStnVNp589gm2fLDVsIySimJuvPYmmp73UPdMC03Pe3oCpu/+1Yd24WnzhhzPbDNDvj8kXADMdjOdeoc8y0iIHJGI/ALjDHvi6cd5/KHH48qvF158gaZjTYarYhtlmDIrWn3HJL9yiIwkDVKqe+7Ln37F8OzHvNERNtHwTz//C2VX28LOiPY8eJQ5X53Wbxm922l0zMbDTTRuamXqReNDyoh0FhbNMVMh1b/PoSAjSdHJhfyC1H7mY8kvMM4wT7uHfc8cZf6XZkUsY6D88nv81Dx1hIk3DI9qJMlV76JxbRszr5wWVb2HSi7kF6RmJEnmJGWoWCZEes1uzPbQZfDNdjPk+Q3PoFw+V8/Qts/ixeKzcsV1l+HydWAzwbGmYwT0ACZlorDEwcG6hmBQ9BpOv9p5BcteeImxV1Vitpvxdw15j50yWp5lJESOi3VCt3GGmVC20H/XYs6vokIqx5QbrodkmGHLG5hx5eSo6y0yn3SSMlQsEyKtfjt+tz9sVIdOc0/npncZnU3e4DX9qyux2PPxdU1YHFU0Bv2IC3uFBZNSoAdobjjGuOHjg8PpPhf5FgdXXbGY0mGlFJUWBecQmNxYA3ZuvO4mtn68dcgncgoh0kusE7qNMyyA7gm9EhJrfjUcaaTYPJxLLru4a65S/xk2ddQ08kssYceU/Mpe0knKUFWLFkaYEBm+GuwV113WM5Gx96jO1c4rWLdybVgZzUdaGXt9ZU8gme1mxl5Vya6/7qUgYKXyjGJMNkXAo1P/QQtTSocZDjVPmDGeO3/01ZBtJRXFaE9pdOa5jt9J0pmP84b4HmwohMgcseQXGGfYwRcbKS8qCxvFjjW/ikqHRVwPqW+GNR9tlvzKMdJJylDdEyKNzn76mjBjPDdee1PYqM6EGeMZO2VMWBl/e+Ahw0nX2P3ofiuNm1p7nomk+6EzxqFmk9VE+ZyikDVJhBC5I5b8AuMMu+n6m3vdrSb5JZJDOkkZLJbVYI1GdSKVEenynM8VYMzFFWHbG5+P/s6O1SveZ8Ti0tBh9sX+pK7AK4RIP7GuZh0pwyS/RDJJJykLxftAyd5D25Zel+cmnTgB3R0Amw5Kga6ju6F4WCHLn34lquOlYgVeIUTmGOr8qhxTHvUxJb9yj6yTlGUS8UDJ7qHtvuuMjBo5iqKCopAHQRbYCqirPRz18YITNv0h27qfkySEyG1DnV9FBUXYzXlRH1PyK/dIJynL9PdAycHofe9I1aKFNL/notBRSFl5GYWOQmqW1THp8tFRH69q0UIaVrb1BE33ZMuqRQsHVT8hRPYY6vxqfs+FHtCjPqbkV+6RTlKWcfk64l6HKGR12+uKe1boPtbQErZ67sjKkRSOKIj6eJFW4E31sv5CiNQb6vy66oqrwB6I+piSX7lH5iRlmUQ8UPLFZ5Z1PdQx9BbaF59Zxp0/+mrIBMXlT78S8/FinbAphMgNQ51fgzmm5FdukU5SlqlatJAnHn2CDmsbAQKYMFHgLeTsqnP408//EvaAWyNesxtvh5nDq5vQdR2lFOVzivGa3GETHGedNIvX3/gnefNNIbfEXnetrBsihIhNpPy66dabIj6ku69Y8qtq0UKqFi3kmWefkQwThqSTlGWONbTQ3HaMEReUYLJCwAuHXmnk+ZeeZ9JNIzHb8/F3rUB747U3GXeU2s0c+aCRkeeVY7aZ8HsC1L3ViLk1r2cBuHybGb+nndff+CfuFjcdm3zHF1fzyiRGIUTsjPLr8OvH2L5xByvWvtU1QjRAhsWQXy+8+ALnnnkeAW+Axk3tkmEiTFSdJKfTWQO0An7Ap2nagmRWKpvEezsrEPUZFASHmsdfOwxl1rEpHY+uGLG4lKYtbRGHn/saNW4E7TPrMVtMoIPZYqJiTjHNh9xhExzz5pvo2OQLe8CtrBsi0oXk1+ClQ36Nv3YYy/78MjO+Ni6qDIslvyrOKeTFp5YZPuBWMkxAbCNJ52qaVp+0mmSh7ttZ+565xDLRr3sSYlRnUASHmk1WO6ZAABSYdR2TzYQy93kQpN2M1+Q2PKat2EL+iEqaGprQdT9KmakcUUlL4cGwCY7KSvDsq3fZsm6ISD+SXzFKl/wKWE2YC5XhKtpGGRZLfpltwTLkodsiErnclkT93c4a7RnKQJMQ+7L67fg6vOTnB29cNKvgqrK6v8+DIN1+rAE7fp+f1//6MqNtZhTBjlTz9lo69ntwu90UWnXavIo2WxuNm1upW1/PyPmVPeXoXkAP7YDJAx+FyHzpkl+eDi/+Nt1wFW1rwB5Wxo5XPqVwvy0kvw421WO3Fxo+0NsasBtulwwTEH0nSQf+6XQ6deB/NE27v+8OTqdzKbAUYNYNU5kyZUriapmGzGYTDkf/f0RePNjzrCHbTHkm3HgHfG83n8WLxR68Pu73+zGbzcFVZC3GZZx56nz++dyrDL+kouehj0dfa8AfULjqXZhsJgKeAIdfP8YXbvsiH72+iRtOmsC1nzmpp4za+mPc+thjmBflYc03UeoK4F+jc/+Pr+UbT77AJx81odsC2Gw22qvd+L1+DpUcpXRqMXmlNpredXHD9VdE3UYjuq6zfd0O/F4f6DqHtuzDHAg+I8l9rI3hhfm0d3pwWS1YbdaQ99lHlFI6qjykPJPFxIwFJ2AyGa96Ec3vM9PlQhsjkPwyMNDnIV3y68gr9Xzm9NPZ8NxHwblKfTKsbzmLTz+JHZ01IfnV9KqLcuswtv5hN/kTrHg8nmB+7XZz07/ezKp3V1JxtqPnmInIsKGUK3/byWpnSXFxxNeUrusRX+zmdDpHa5p20Ol0DgdeB+7UNG1VpP0f++hBvb29fTB1zRgOh4OB2rj86Vfwzw2/tdS80RFyJub3+XnvqZXobm9YGW+v+hDbDDPKDHpAR5kUuh882/2ce/bpeAM6595xAWZL8BjvPPwia3d+jK/SFOwC+8B0xIcvYMEy1oxu1lF+1XPHSHFFMSsfep1Sv79nJKmtw8Vra9ZiKVIETApTQMfkVvzzJ/+CUopf/eN19tfvY1zlBH5w5WJ8/gC//cfbbD/UiO4zMWzECMqGx75uiMfloQQdu81CIKBz5qThjCwNrsE0ZXQFFSXR/XF8UnOYNlfoMHxDq4u3dx7G3Oey45FWF7bSQmw2Gx6PJ+Y6Z5JktvGJB19WA++VGrHm1+3/crWe7Z8FGPjzsGXrTryj3Cjz8RML3R/AesjO7FnTojrGQPkV8AeYds5sxp4wDjDOL0t9gFljp7PbdZQOa1tYhvW99Pf43Q+xvWZHSH55W3RevPsLlBbmh+TX4hknom3dj9VqYXfNfjwBLzaTlSkTx5HvyIv5Z+rz+pl31RmUjSwfeOcEiubfo2yQrHZOKp7KhbOXGGZYVJ2k3pxO5z1Am6Zpv4u0j3SSgnpf0+8+Q2lY2RZ2Tf/NB1/jewunM21sZVgZG3fVsvTl5xl+eRl5+SY6XQGOvNTE/ZdezdypY/jPZWspuvR0Crs6EEYds8bDTRxe3YStyIq30401z86IkyoorCk3HDZ/8s+Pk3dCI/W72vEDZqB5XydVwyfxxzuu5r8feZq75nZw78YCvnLb9Sil0HWdR15+l9suPQul0vbfy4hyIWSS2caSi3+YEb/0aPLL9859WZ9fMPDn4WBDC99//VWKzynEYjPj8/hpWdnGry+4mNEVkc+8exsov5rbXPx41TbOvOEcIPKJZc1TRxhxUQmHP24YMMOM8svv1RnTUMpj37rZML+AhGTY+k8P8A+vzsyqmYN6/2DlQn5BajpJA6647XQ6HU6ns6j7/4ELgS2JrWJ2imZ11o/f+ojPVBYZdpAA5k4dw9dnzMf9yEGanziE+5GDfGPGAuZOHQPQM/rTzWjF2vnLf5wAACAASURBVJYDbbQcbMU6Cgqn2rGOgl3/3Mv+PQc4sLOWAztr8Xl9Pfs31h9l19Y28qrKKF5UQV5VGQ2HPVQ31LNm627OrGxFKUVVZStrt1YDsGbrbtwHtvR8L0Q6kPwavNEVxcEO0bo8rO/4Gb0uL6YOEgycX0BIhyTiituedvavPUTpafmMPLeE0tPyqXm3tifDWhpbevY3yq+6/Z3UHjsWMb8gMRmWEWcKIibRzEkaATzvdDq7939C07RXk1qrLNLf6qxH9x3FvW0ft926KOL7dV2nrW4/q68Z1jNic+/Gfej66YZnO71Xj23e3cLBVYc5dqSNklMceBq9mJWOX1cUjbfT+X49U3Yf5FDDMbbUHGbOBfMACJhtjFpciNluRgHKAuOvHMGOJxp41LuCB5YEQ2zhODP3rl/PgpmT2LB+A3dV5XPv+vWcOmtyRo4miawk+RWH0RXF3H35eYN+fzz51c3v8eNu8zHpnODSAFYVvFfE0+ah81Aww15cdoCLvnU1YJxfZScXY9qoB3Nqbmh+nTprMoBkmDA0YCdJ07Rq4JQhqEtO8Xl9fPj4Wzz8lYv63e/4mU/wV9X7DOi02eGTS0+tWsDDf3wU8r1UjC7gtOvHs16rofKUEqzFCpNJEQjo+NpgQqeDq6tm0Nzm4rJ7HqVi3DDGzRhH5ZhyfO42sOmgFOg6RWMdWBbk0bKnlp+/aWZMsYkvnF5KVWUrj738HmdWttJ4rJOqSn9I3TL9MpzIbJJfqRVrflUtWmg4RWH8CWPwu1zYHQq9a2kAgHlzx3HVwhN44PV1PatrG+WX7oX8AgtnVrYCZh74oKknv9ZurUZHlwwThpKyBMCqJ97C7c7uiY92uy2uNjbU1PHLaxdiMZv73W/nnloC7WV89MnxbboO5pranj/iloYWDu6o5eAn+9CPHuNnF5zNxroDNAfclG3Mo3TsFHZ6GzC1+fDpYFdgdZsZk18EwKc1tdw02ceeZe/x6at5tB1ro2KBA3eHmwA6AbdOw/stdGxsZc7kSlxWxf7Gw/xolWJMRSnb937Mreea6Wx3cUqlhb/2OhM7PoQ92jAUhRDZa6D8sphN7N5cHezMdBmrRrPrb9U9k6inTp3Mrr3VtDU3YjbrBHQdk1J469yMmVTEmq278dTtZ9l9z1ExZjjHdjbjaevE4/Oio6NQuA95GEk+m1oreGllK9YONz9apRhbWYZpzwFaGw/zlRm+uDPMbrOw9c0NNB9oSNaP1Pi4cf57lCmS1U73yR1cOHuJ4WtJ6ST94pxZdLR3JKPotFHgKIirjfnnzcZuO/7jf3HNDg40tIXvaCmCkiLDMh5b8TGXzZnIq2s/5ZTyQr5+wcmUFgZvt13CCT37vbz6I7a/8ynDLiglL0/R6dape6eeOXNPQteDQ9B3zA7w8J52vnzrJby85lPu/sNrFJ1cREeTC7vFwihLAX/5+WcZVV4UnPi4aAT3bsznhNknM7d4FS5XKyMKTRzucFFVaWHt1mpOnTVZhrCFyGG3XH5Ov68X5tt56s7L8PsDoS9cNCfk23+WWXh087uMOLcEu03h9ugc7QxwSmUlG9ZvQLu+kId213NH1VnUzRzNT99+g6JzhtHq7qTInkfb6nbuvfTS4/l1TjC/bv3s1azZupt80y5cLlfcGTZr0kj+9rnz0AOx3RAVr3j/PcoUyWqnGjkh4mtJ6SSVFRVgMw3th2SoORyJbeML22o5/cb+A6WvA4ebuWfZGr505gzOOWlixP2aGpo52VvI5kcPQoEJOgKcUjGcpsZm1mzdzdzCevB1MsdRz7pP9nD5GScyf9oYfvL35RxxtzChZDQ/uHIxoyuK+XDLrp7h86rKVl7+cAt5PgsrOrwMK1Ac7fDjL7AwXK/tGcLu3jfSELsQIncVFYQvCNmXp6OD+aqYzc8e7smweRXD+XjbbhaV1pOHm4Vlzezcd4jTZk/h95ddxq/+8Tr1DQcZVjGen155qWF+rd1azc49tRypTVyGlQxi6YB4Jfrfo3SVrHZ68yJ/Bge8u00Mjbx8OyWVJTF9TZw1gUu+fQ3/aGjn8VVbI5Z982VnM6KsiEfPy+P18608el4ew8uKuHHJWWxYv4GTyt0MLzRxcoWb9evXo+s6o8qLmO8w8fZllcw0+xhVXtQz6nTGuOMTH8usXvLy8vj+4mF88cxhfH/xMPLy83vK7r1vd9lCCBEL4wwrpNjsiZhfM81+3rikot/8Wr9+PTdd+hny8/Mlw4Qh6SQNAV3X+b9l7wz6j6v5aDPLn36FZ598luVPv0Lz0eae18wWM6ddcQYr6prZXWt8Hbx7tKjIEqC+zU+RJcAcRz2PLX+XuYX1FFr8PLCuk0KLnzmOetZure6acNnCvrp6qipbem1r7RluVkoxx1FPhe8oAA980AQQMpm79769b7mN92cihBga8f6tHmxo4ZcvvcV3X3iZX770FgcbWgZ+Ux9GGTbDWofDdTCu/KqqbOWx5e92TeiOPsMkv3KHPLttCEQz8c/tMn7YbLQPmZx94Tzu/K8XWf6jz4aVsaO6lk8/dbNmj5eGNi8VhVaOeEy4rNXs8Lp5e4cXC35+9a6XNt3NCbb9tDUd4fZJnfg63UwvsvLw+vUUlg1H7widhFl9wEWLW7FrpQqZDLl5fzWMizxhUyZ0C5EZ4vlbvfvxlby65WPyJtgwmRUBv84zv/uEryyqYulF86IuxyjDdjZ5aHHDwWYGnV+6DpsPVMPYsrAJ3f1lmI4u+ZUjpJOUZN1DvEYT/3rfWnr17PE8/Z/Pc+6Xl5DX6xr9QA+ZrNt7mBf+4+9cvGAGf7jdeD2T6ZPHcLJpJ2uqvfz+kgJ+/6Gf62YUsVmfwinqU9ZUe/n26TZ+/6GfiycXsVk3c2ZlC52uDsYUmqht66CqsgX3hPmcNntR2C2xuq6HTYbsb3Jjfz8TIUT6iDa/Iv39vrOzhhO/NS1s3aO///kjll40L+rb640ybOGkfCaVmthysD2u/OpuS7QZ1rOv5FdOkMttSRbtCq+XzZ/C7649g7f/sgxXe2fPPkYr0JosJmq27eXV+55n79/f5upRHZw3fTjTxg4zrMPOPbW8tMfK9Aoz9Z0WppWbeXmPlc3bqyNuf267jwc3uPnLWjcPbnDz/HYfO2tqw+o9UBtj/ZkIIdJHvCtUewIewxW03QFf1GWAcYa9We3n8Y+9cefXQO2M5Wciso90kpIo0kRBXdd7XvtmVX7PtmGlhfznDWex8q8vU38wOL8ouAKtHwD3MQ+179Sx5X93MMOex0N3nM/0/HZ+cI6j3wmFN192NhXF+VxyUgXl5aUsObmC8uJ8fnvXrYbbf/PtW2hra+erp9r4+hl2vnqqjba2dm5cclZYvQOBQEyTG/v7mQgh0kes+WWk0mZn11PVVD+9p+dr11PVTKwsiboMMM6w+dNHMWvy6Ljyq3dboskkya/cY77nnnsSXmhg3wf3eL3hT7TPJjabjYHauGbrbiZ7dzCuNPgHpZQCbyf7PCXUHm08/lrXtjHDy8m3W1l0wmhe+8eHtCrFCXNO4MMX1nNg7UE697UxYnwBo5WDn19xEduqDxiWEW09Xt/RwWmOQ4wtMfGnFYc5fWIhytfJg+/s44JhR5hRYcJmDj4dzqr8vHMA3F5vyDFf39nOaY5Dhm2MpS6R9h8q0fw+M10y25g39eyfJqXgFMiF/IKBPw+Dya++zp01iQ0HDzDhwkrGzi5j+JRCHIdN/Obqi6POr0h12ba/iRNLXEyqsA46v8LaQv+ZJPmVWslqZ6BgGAVjTjTMMJmTlESRVpvtXuHV6BlCSilKC/P5xc3n8LsX17Dq9Q3MdQwjf0QpbluAssY8br9gAaPKi/jHslcjlhFNPbonJj62/DDFXg93Lncxa+IItu2oocEWYE/T8QXePjoC5pLdtDccCjnmU8u3oE8YwUefqJCye68IHk1dIu0vhEiNweZXb90PyX149Tqa/J0MN+dx9wVnxZRfkeqy6XA7O46ZeG6na9D5dW8/E7qNMknyK/eoZAwT+t65T29vb094uenE4XAw2DZ+uGUX+XtXsnD88T7q6n0+3BMWRf2HlogyAAKBAN/7+X3890U6X3lN8dsffZO1n1T3lG02mfEH/Kze5+MjfRpz1M64j5mO4vl9ZopktrHk4h9mzczVXMgvGPznQfIr/eRCfkHy2umtOJHKU68xzDAZSUqBRJyNDFRGtHeNPPryu1w/zUtDu4nrpgV4bPl7mPQAgfYyNm3VeX3DHi6YNwlQPbfKylmUELlL8kvkEukkpcBAzzNKRBnRrG0SCAT4eOM6bl9kosMTYOEYE8+sWMtvf/RNTCYT//fSKmaXbYfC4dx22WfirrMQIvNJfolcIne3ZaHuOzBunekPufOi7yqx3WdhHV6dEYWKDq/OddO8PLb8vWAAbVjLN0+zsnn9GgKBQH+HFEKIhIiUX92vdWeY5JcYCjKSlIX6PrS2+6GMfc/ONm+vZmuHmXf2uqksUNR36Bxx29Ed1Tyq61w+2UtAV1w2KRg8cjYmhEi2SPnV/Vp3hkl+iaEQ9UiS0+k0O53OjU6nc1kyKyTi030W1vehj93rGfVeH+R33/kckyaO5ydLxvC1c8fwkyVjmDRxPL/+1i18vGEtp45SjChUnDZaydmYyGiSX5khUn4Zrc30H3fdJvklki6Wy23fALYlqyLZLNLDEGN5SGK0ZfQ8CNIaQAFF1q6H2XY9rLHxWHvPKrGRHvj4q4de4vLJXopsClAU2Y6fjQmRoSS/Bikd8qt3XnVnWPeDaSW/RDJFdbnN6XSOBS4FfgF8O6k1ykKRJiHG8uDIaMvofhBk9VEdhY4ObG3oJGD+mFvPNdPZ7uKUSgt/7Wd9kG0799BgNVhnpHQ3yJC1yDCSX/FJdX590nj8obVfmeHrybCn3jZeo03ySyRStHOS7gO+BxQlsS5ZKdIDImN5yGv3vrfP9PPwAGVMnzyGU8y7OGOcmT+tOMydi0bw4Jo2JpWBy+ViRKGJwx0uqiotPQ987OuE2bsM1xlxT5CAERlJ8muQ0iG/3t/v56OAmTMrW0My7IaTinBPmBPWQZP8Eok0YCfJ6XReBhzRNG290+lc1M9+S4GlAD+7ZARTpmb3uhMmkwmHwzHgfqs37eDsEe1YzFY+M7ydj3cfZOEp0yNuj1TGvKIG8HUyt6ih3zJqao/idw3jyVcO4fB4+MZrbnx+Kx8ddrOuwEtlgYmj7V70IjsjrUc49/STw47XXcbWHQAK0NF1sBw03j8bRPv7zGS50Ma+JL8ii+bzkA75ddKkUWzdX0NToZ1VbY0DZpjkV/ZKVjv1kpKIrw244rbT6fwVcCvgA/KAYuA5TdNuifSeXFixNpqVP3Vd578feZq75nb0nDndu7GAL996Hf/z6DNh279y2/VhZ2PdZdw2vpaRDp26dsUj+8b0W4au6yGr0P7mh9/g/seejep4g2lnNsiFdubiituSX5EN9HlIl/z67Y++iVLKsC4DZVgu/F2DtDNe/a24PeDEbU3TfqBp2lhN0yYCNwBv9Rcw4rhIE6MjTThcu7U65onYRmV0rx9iUorrpnn59UMvRdxXiGwm+TV4g8kvCJ2MnYj8emz5exHrIhkmkk3WSUqiiA+W7Wd5fB09qomMLutuGBdehqo+wMeb1vO1i4K/2sWTLfztuW1sdEyP+iG0QggxmPzqux5bIvLrmdfWgn8euOSRImLoxdRJ0jRtBbAiKTXJQrEu36/rOn/9P40yUyfr1q0LmcgY/lDGzxiGw/+9tKrrLMwKgEkpPj/HxI6i4dwqd3aIHCb5FZvBPH5E13XWr1vfk2Hz5s+LO7+um+Zlh9nMbTdcM/jGCDFIMpKURtZs3U257yhtHZ2Um46ydmt1zA+T3Ly9mj1+O6/tOz5qFNBNtFuq5fZXIURS9c2wtz7cwthCyS+RuaSTlCa6z8AsHR3ctdDGvR92sG7dOv7lc84BJ1f3du93b09eJYUQIgKjDCst9nDrZ6PPMMkvkW7kAbdpovsM7MwxXZMSR0O57+iQTEyMZeVcIYQwkqoMk/wSySSdpDSxo/oAm+q8TKmwcdRlZmqljU11XnbsOZD0Yx+faCl3igghBidVGSb5JZJJOkmDoOs6Dzz/VkLPXKZPHstNc4upqCijvLyUiooybpxbzPRJYwesSzxnUX0fGilnY0Jkv3TJMMkvke5kTtIgrNm6m/aajazdWpmw209jnaDduy7RPj8p0vuD649YetYdkVtqhchu6ZJhkl8i3UknKUbdZy7fq3Lw2wGeWRSLwd5uG+3zk/p9/1wzAAvHmQdVjshdB+tb6Oj0MD/VFRFRS5cMk/wSmUA6STE6fuZiT/mZS7xnUb3fD6Gr2MrZmGhuc9Ha4e75fmdtM2t2HSWg6+xt9pPvcOCyllFYOYoHUlhPEZt0yTDJL5EJpJMUg3Q6c0lEXQZ7iU9kjmPtnXi8PgDaPQHa2tp5ee0eWjt9IfvV1Heg8opDtrX6rTiGj+/53lY0nQmX3AHAfHuenK1noHTJMMkvkSmkkxSDRJ+56LrOIy+/y22XntUTDEbbklWXwVziE0MrEAjQ5vKEbNt9qIm1O4+GbGvr9LCnOYDNZgvZ3uyzUVA+EgC7DdweL8NnXE1x5ciQ/abZ87Da7ElogUgnicywSFkVTYZJfolMIZ2kGPQ+c7FYLPh8vrjOXIwmLUY7kVHOojJLIBDA5w/0fL9h12F2HGwO2Wf7wVa8ptCOSnN7J9aKCSHbbIWljJ13OXD8HyCl4KSySnkiuuhXIjMsUlZFk2GSXyJTSCcpBr3PXOL9B8do0iIQ9URGOYtKf+s+PUh1XTNr9jTT7FaYS46P3jiGjWP4CZeE7D9yQQn5jqKhrqbIIYnKsEiTrqOdjC35JTKFdJJSxGjSoo4ut7NmsJUf1bBhTwMHjvmo99opn3giFZPPY9L8Udjy8lNdPSESJtKka7klX2Qb6SSlQKRJi7qu8515qZ8ULgZWe/QYH+2u463tDbR7Avjs5ZROPoWKBVcwY2T/C4AKkcki5deCmZPSYlK4EIkknaQUMJq0OMdRz77mAEoV9WyTM7H0EAgEWLNtP7UN7aypaeFIB9jLx1Bx4nlM+exMLFZrqqsoxJCJNOn6seXvyi35IutIJykFjCYtVh9w0eJWeD45PidFJjKmRkenh09qDvPqpoPsb3ZDfhkFk+dRUDacKQtPZqqcFYscFmnS9eYD1TA2tydjN7V28B/Pb8JkNg/pcW02Gx6PZ+AdM1yy2jlnAXzhVOPXBuwkOZ3OPGAVYO/a/xlN0/49kRXMNTJpMb3sOdjAlgPVvLftEA0eG1jzKJ1+GqPOv44FZRWprp6Ig+RX4kl+has+2ECH28vWPXX4pl3AtFOH9meUK3euJqudw4YVRHwtmpEkN3CepmltTqfTCrzrdDpf0TTtg0RVUAws2vWTRP/aXG72HW7m+Q9qONraic9eRqBoJGNmnsuUm2eRG+e7OUXyKw1kc365PT6+9dhmpi+8AN06jkknnZbqKokEGrCTpGmaDrR1fWvt+pJHLQ+xeB8EmasO1rdwqKGV1zcfpKbJg99aRNGEWUy46BoqHUWYTCYgd87Eco3kV3rIlvy694WN7G8P/WfT7/dz0vlXMuGUs1JUK5FMUc1JcjqdZmA9MBX4s6ZpHya1ViJEvA+CzBWBQID6Yx0sX1fD1tpmOlUB3sJR5JeOYPJFNzE3L7+nUyRyh+RXamVTfh1u15n12X9LdTXEEFK6Hv1JldPpLAWeB+7UNG1Ln9eWAksBfnbJiPlTpmbu2UI0TCYTgUBg4B0TYPWmHVh2/5OF46ys3uclMO1CFp4yfUiOPZTtjJXL7WHLniO8t+0Qn9S58AQUeSOnMvLE06gcNwV7fuTrzH2ZTWb8AX8Sa5t6yWzjv5w7Pe3/1ZP8CjVUf9vZlF/ffvhDTrzlpwkrL1FyIb8gee2cOaqYc2aMNMywmO5u0zSt2el0rgAuBrb0ee1+4H4A3zv36dl+6WKoLs/ous7q1e9x11yFP+DntDGKe997j5OmjB6Ss7F0uQwVCATocHt56cPdHGxop7bdRJPPSsm46Yw95XPMWFQRMkrkC+j4Yqh3urQzmXKhjf2R/Ao1FJ+HbMsvt8eTln9DufK3nax2HjumAyMNX4vm7rZhgLcrYPKBxcBvElpDEVGiH6qbCfbVNfHhjjr0gM4He5qx2PJobHNjLh7JyFOWUDijghMrR6S6miIDSH6lVi7ml8gu0YwkjQL+r+u6vgnQNE1bltxqiW7Z+iDIfXVNrN9Zx7u7m7Ba7bS53HRYSjBbLJjzChkz9yqUUkw9dTRWe16qqysyl+RXCmVrfoncEc3dbZuBuUNQF2Egm9YkaWnv5J/rdrPi00Y6i8YyZu4Spi6YKitWi6SR/EqtbMovkZtkxW2RVH5/gPe21PDk6r14HSMZMfdCTr7j5Iy9u0UIIUTukE6SSIrVW/aifbCXFt1B6YwzmX3rF7Dl5ae6WkIIIUTUpJMkEkLXdTbvPsSz7++htsOMY8LJzLp16ZA/w0gIIYRIFOkkibjsP9LM8+/vZuO+Voqmn8HUKz7LOEfRwG8UQggh0px0kkTMmltdvLJ+D29/2oSpYiITT7udhUvGyjwjIUTKffkvb5NfUgkk/qnxPltlwsoSmUE6SSIqLreXp9/dwZrqJjz5lYw6+TwWfH6udIyEEOmleBSzPvttIHcWWRTJI50kEZHb4+OdrTtZtq6Gel8e46uuYvaZJ2C2yC37Qgghsp90kkSIQCDAlpqjvLimht1tNibOP5dx19zE1BiegyaEEBBcAsQ/hM9+1PXgTSRCJIp0kgQAhxtbefa9nazZ30nJpJOYcP7XqSqrkOFqIcSg3fz7f1Iy7oShO6CuM/7UC4fueCLrSScph3V6vPzvqx/z8cEOKB3LhDNu4cxLJqa6WkKINNXU2sH/0zZg77PmmTng4Re3Lgzbv3L0eE657mtDVT0hEk46STmmudXFq+t2887OJjqspUw8y8mpS6anulpCpI3vPvw+7gTeEZWu7DZbzO1sbm2jsuqzjJ85P2T7x68+wfcfX4uiz40ceaPiraYQKSWdpCxXe/QYew83setQC1sPtVPnzmN81RWcctZsuTNNCAPTb/lZTlxiTuSl9JMuvikh5QiRbpLeSfL7A2zefShsMt2W/U3sONwRsk3Xoa7NjxUflUX2sLICgQAXzR5OUUH4a4UFdqaPG5bYymeITo+XT/Ycpr3TwyubD2MymTjS6sVaVE7AWkjx5PnkjStm6rknMjXVlRVCCCEyRFI6SV95eFPPAl4er4+CSfOw2vJC9ikYU8mERbPC3juxn3L9Pi8vf/Q+dIbfvdD+6QHUyi2GoyPe1gaGFdnCtuu6zvmzRjC8NPzOLbvVwtSx6bVw2O7aelweH69uPMAxl4+mNjc4KnD7ApRMPQ1lsTD5ujOwWG1MSHVlhRBCiAyXlE7S/FvuTspwtdliZdr8sxNWXiAQYPm6leit4dflO5rrMb+9GZPJFPaap62JCZWO8Ov5Opw5YziTRhSHvcekFBNHlQ94iauxpYOm1g5a2t28tH4/bq+Peq8dm81OoGQs+aVjGH76xYwfMZrxsTVXCCGEEDHI6TlJJpOJ6aedG/P7dF2nID8/rCOoo/P6+nfwbm8Ne4+now29YSNWa/iP3N3eAoDdUUyHbqdw1BRQZqZcfgsWq42J8pBYIYQQYsgN2ElyOp3jgEeAkUAAuF/TtD8ku2LpTCmFyWw2fML91EF0ugJ+P4BheUKIwZP8EkLEI/xaUjgfcJemaScCZwBfczqdM5NbrdwSqcMlhIib5JcQYtAG7CRpmnZI07QNXf/fCmwDxiS7YkIIES/JLyFEPGKak+R0OicCc4EPk1IbkfXeevk5mtvdYdtLHXbOu/SatC1bZD7JLxEvya/cE3Unyel0FgLPAt/UNK3F4PWlwFKA8RfewZQpUxJWyXRkNplxOByprkbSJbqdbZ1+AjPCn63UtvOtuI8TT9m58PvMhTZGIvkVLhc+D5Jf2SVZ7SwpCb8jvVtUnSSn02klGDCPa5r2nNE+mqbdD9wP8Ne3d+jZvmJtrjz4NdHt9Pl9hk8F9/l9cR8nnrJz4feZC200IvllLBc+D5Jf2SVZ7Tx2TCd4b0e4AeckOZ1OBTwIbNM07feJrZoQQiSP5JcQIh7RjCSdCdwKfOx0Ojd1bbtb07TlyauWEEIkhOSXEGLQBuwkaZr2LvR9tLMQ/Ys0CXHXti2UVM4L2+4/sC/uY9Yd2Ie5sjFs+7FtW3hOezJsu0yIzH6SX2KwjDJsxydbKSyeGfaYLcmv7JXTK26L5Glud+Odvjhsu3fjWtp3rA7bbvOFd6hiFfC56TQo29vpMqxL84434j6mECI7GWWY6Ugbx9a9RH5l6CoSkl/ZSzpJYkg5yoYx6ozLwrZbE/AHP3riNMMwOdRSF3fZQghRMvNs2nesZkyfDJP8yl7SSRJheg8zW8wWfH4fAI21eygfMylk37oD+wj43IyeOC1k+8GanQwz+INvOXIQ96rwG4zcNRsN6xLLkPLBmp14GjvDtruajkb1fiFE5oslv/rbbpRhTR+/jbf5MDV9MkzyK3tJJ0mE6T3MHDCZem5Lbat5gKI+oWGubKRzx+qwMyBfTY1h2bq9AMe8y8O2dx6ujntI2YeVgpMvDtvuOrQr6jKEEJktlvzqb7tRhumBAMULnZgLSkK2S35lL+kkiag11e3H9fbTIdsC6PgaD0Rdhq+lnsa3Hgzb7nd30NwYPmnRaELk7370bdzm/LDt7uZ6Ktubw4/pjn6+QKJWvZXVc4VIL0b5BdBZt59RUZbRcWAr3pajoEJXz4klv8A4wzxuN7YPX6Bo9qKQ7bHkFyQmeyS/jpNOkoiaynNQdOpVIdt0v4+mFX8L29dmysAdbAAAIABJREFUMRlepzc5yig79/Nh2+tf/k/0vPBVTz2+8MXV3OZ8Si74mmEZZnth+DHNZsO6lDrsYdsiTTiPdZJkosoRQiSGUX4BuI/uMdzfKMNM1nzKzvsCfW+YjCW/wDjD/O1NtG9bFZZhseQXJCZ7JL+Ok06SSIqRY8dzjfPGsO2bN21AKaM7shNwl7YCsyX8I11UUmpYFyGEiMQow4L5ZbQGcyJWmVCACsswya/Ukk5SDos0pLr5/ZXYa5vCtvtdrWHb2ra8ie7ppPaDZSHbPXs3YfT8B7+rFb1rImUoHa/XE7Il0NlGc2MD9/32VyHbfZ0ug/cn18GanbJWiRBpJBH5FdzeFpZfYJxhekCn9aPXKJx9ft9XwvILoKn+cFh+wdBn2OF1rxFo3B+WYZJfA5NOUg6LNKSqr19Dwaxzw7Z31u3C33EsZFvA46J4oZOCgtCHDnrqdhuWbdqwHkzhZ2IBdxttH78Zss3bWIt1xBTyTg8dIm9eZvx0Cd3vo2Pzq2HbbXgN94+FD6sMPwuRRhKRXwAmewGO6VVh240yrLTgBDr3bAjLMKP8Agh43eSddXt43SNkmPfo3rAMS0h++bw4FlyDt7w8tB6SXwOSTlIKZOykOD0QdleHv60RlMLjCW2P1xvhD1sp4+HqgI63qTb0cB4XYHxN34jJYgtbvwSg4a2/ySiQEAmSTfkFQCAQll8QIcOUAmUKzzCD/OouO1qm/EJMtrywDJP8Si3pJKVAuk+KU2aLcZiYzBx7/c8hm/weDxZHGcpiC9keMRp8bhrf/N/wom15lJ39udAyOtto2/p22L4BV2tYPQBMnccMJzgGfNH/vEsddsPtNsuAz4KOqpxIky2FyBTZlF8AyqRQhZVh2w0zzO/Dc2gnjQ37Q4s2yC+A+lf+aFhHowzz+7zgagnLsFjyC4yzx9RSh9kU/bwpya/jpJMkomYyW/jhT38Rsu0X//7DsA5Sf2wFRVRcsDRs+6GXfo8y9/k4KhX86qN81AS++b0fRH3M57Qnox6wjnRmFksZ/ZUjhEgNo/yCYIZFy2KzYx89jdK5l4RsN8wvMMwviC3DEpE9z2lP4i0ti6uMXCWdJBEXv8+LHjCYiK1HP8ycKnUH9vUMY/demVeGsYXIHZmaYZJfQ0M6STks0pAqrlZa174Qtll52sO2BdwdtG0OL8PX1mx46cvud9H57sPhZesBdH+f8yU9gPdwddj+hVY9vM6D4PEFDFfmjeUyXC4OPwuRDhKRXxBbhrm2bcHf6aKz/XBo2Ub5BeD3GuZdIjJM8mtoSCcph0U626g7sA/zKcFbXJVS6HrwD9qvh99Ca3WUUDQnfCn9wNHdxmt7RFjv477f/gqrtc9lO2s5peUVMV1aSxY5MxMivSQiv2AQGWbAML+AssoRkl8ZTjpJKZDuvfqAz03njtUAmJQi0BUyNl/4HSBmXyftG14y3B6LQqtOW5LOuCL9vC0JuLVWiFyTTfkFickwya/sNWAnyel0/g24DDiiadrs5Fcp+6V7r370xGk9w7jmXsO4RpfPpp+ywPDOC6N9+/PFb909iJpGJ1GTsUVmkgxLrGzKL0hMhkl+Za9oRpIeBv4LeCS5VREiVMauxyLSzcNIhokUkAzLfAN2kjRNW+V0OicOQV1EBkrm0Huy12PpXfe+d4eI7CEZJvqTqRkm+TU0ZE6SiEsmnw31rrvD4aC93fjuFyFE9srUDJP8GhoJ6yQ5nc6lwFKA8RfewZQpUxJVdFoym8w4HI6Bd8xAw0qLaNq9Agi9O6SstGhI22wxWwgYPOfNYrYkvB7Z/PvslgttHKxcyy/I3s9DuuQXDF2GZevvsq9ktbOkpDjiawnrJGmadj9wP8Bf396hZ3uvNpt77p+58Pizg/q2cyjb7PP7eiZd9t2e6Hpk8++zWy60cbByLb8gez8P6ZJfMHQZlq2/y76S1c5jx3RgpOFrsT2QSgghhBAiR0SzBMCTwCKg0ul0HgD+XdO0B5NdMSHSfT0WkRkkw0SqSIZlvmjubotuyVEhEixTJ1SK9CIZJlJFMizzyeU2IYQQQggD0kkSQgghhDAgnSQhhBBCCAPSSRJCCCGEMCCdJCGEEEIIA9JJEkIIIYQwIJ0kIYQQQggD0kkSQgghhDAgnSQhhBBCCAPSSRJCCCGEMCCdJCGEEEIIA9JJEkIIIYQwIJ0kIYQQQggD0kkSQgghhDAgnSQhhBBCCAPSSRJCCCGEMGCJZien03kx8AfADDygadqvk1orIYRIEMkvIcRgDTiS5HQ6zcCfgUuAmcCNTqdzZrIrJoQQ8ZL8EkLEI5rLbacBuzRNq9Y0zQM8BVyZ3GoJIURCSH4JIQYtmsttY4D9vb4/AJze3xtmjCjiWEsgnnqlvZLi7G8jSDuzSS600YDkVwS58HnIhTaCtDNewwqtEV+LppOkDLbpfTc4nc6lwNKubx/TNO3WqGqXoZxO51JN0+5PdT2STdqZPXKhjQYkvyLIhc9DLrQRpJ3JFM3ltgPAuF7fjwUO9t1J07T7NU1boGnaAuDEBNUvnS0deJesIO3MHrnQxr4kvyLLhc9DLrQRpJ1JE81I0lpgmtPpnATUAjcANyW1VkIIkRiSX0KIQRtwJEnTNB/wr8BrwLbgJm1rsismhBDxkvwSQsQjqnWSNE1bDiyPodz/z96Zh7dVnfn/cyRLli3vaxY7+0YSyMaWBEhYwtYQAiQCwlLoQvm1A2WGTqd7O9N22s6UFrpQYKAFSoFewhZC2CFsCVmcjQRC9s2JE2/xbq3398eVHdmWbcmWLMl6P8+jJ9G9R+e+5+rq63Pe8573DPq5UZKjjSDtHEwkQxu7IPrVLcnQzmRoI0g7o4bS9S4xjIIgCIIgCEmPbEsiCIIgCIIQhJCm20LF4XD8FVgInNA0bWok644XHA5HKfAkMATwAY9omvZAbK2KLA6HwwZ8AKRiPCPLNU37aWytih7+rMwbgXJN0xbG2p5o4HA4DgANgBfw+FdxCQGIfg0ekknDRL+iS6Q9SY8Dl0e4znjDA9yradppwLnAtwbhNgdO4CJN06YB04HLHQ7HuTG2KZp8GyOod7BzoaZp06WD1C2PI/o1WEgmDRP9iiIR7SRpmvYBUBPJOuMNTdOOaZq2yf//BoyHc3hsrYosmqbpmqY1+t9a/K9BGbzmcDhKgC8Bj8baFiG2iH4NHpJFw0S/ok9Ep9uSDYfDMQqYAayLsSkRx+/CLQPGAX/WNG3QtdHP/cB3gcxYGxJldOBNh8OhAw8nQ3ZeoWcGs35B0miY6FeUkcDtPuJwODKA54F7NE2rj7U9kUbTNK+madMxMhSf7XA4Bl2MhsPhaIs/KYu1LQPAXE3TZgJXYEyxXBBrg4TYMdj1Cwa/hol+DQzSSeoDDofDgiEw/9A07YVY2xNNNE07CaxmcMZqzAUW+YMCnwUucjgcT8XWpOigadpR/78ngBeBs2NrkRArkkm/YFBrmOjXACDTbWHicDgU8BjwuaZpv4u1PdHA4XAUAm5N0046HI404BLgNzE2K+JomvZ94PsADodjPvAdTdNujqlRUcDhcNgBk6ZpDf7/Xwr8V4zNEmJAMugXJIeGiX4NDJFOAfAMMB8ocDgcR4Cfapr2WCSvEQfMBW4BPnU4HFv8x37gz+o7WBgKPOGf0zdhbOWwMsY2CX2nGHjR4XCA8Zt/WtO012NrUvwh+jWoEA0bPMRUvyTjtiAIgiAIQhAkJkkQBEEQBCEI0kkSBEEQBEEIgnSSBEEQBEEQgiCdJEEQBEEQhCBIJ0kQBEEQBCEI0kkSBEEQBEEIgnSSkgil1CillK6UOi/WtgiCkDwopW5TSnl6KXNAKfWjgbIp2iilHldKvT2A1/uZUmpPp2NLlVJ7lVJevz3z/X8DSgbKrkRHOklxjv/B1pVSvw9yTldKhZNh9TBGkrWob/Tot63t5VVKHVNKPaWUGhbtawuC0BWlVJpS6udKqd1KqRalVLVSaoNS6u6AMo8qpVbHyMSzgC46FwmUUulKqR8ppbYppZqVUjVKqXVKqbv85x5QSpUrpYImWPZ/7qmA9/lKqf9RSn2hlGpVSp1QSn2glLq1uzoGgN8C5wbYaAb+CmjACODbwBqMvwFHY2FgIiLbkiQGLcC3lFJ/0XV9V18r0XXdC1REzqxe+ReMPaLMwFjgz8DfgYsH0AZBEAz+AlyI8cdyK5AFzMD4AxpzdF2vjEa9Sqks4H1gGPATjEFiHXAmcDfG4PFh//+/BLzc6fPnAqdj6Bl+L8zHgMdf32bADcwBvgNsA7YwwOi63gg0BhwaCmQAq3RdLw843q+/AUopBaTouu7uTz0Jg67r8orjF/A48DawFni50zkduDng/bcxfpyNGD+EZ4GhAedH+T9znv/9x8AjQa75GfDrgPc3+OttBQ4AvwPsvdjdwTb/sbuAuoD3Cvg/YC9GR3Af8N9Aqv/8GMAHzOlUzzz/8TH+9xnAA0A50IwhWtd2+swP/PU7gUrgDSAt1t+vvOQ1UC/gJPAvPZz/mf93G/i6zX+uR23xlxkLPAfU+H+H24CF/nO3AZ6AsjbgBWAHUOI/dgD4UUCZAxh7dD3gr/M4hrfEHFAmDXgEo9NTCzwI/ArYE1Dmj359GR2kzQrI8f//Q2BlkDKPATsD3r/ivwfZQcpa2rQRv3YHnJsJvAac8N/HDcDlnT5/tV+/mv3f13pgRkDdvwOO+HXsGPBsp+9vT8D97vxdzve/9LZ77i87DmMwe9J/D98ETg84fxtGh/BCv22utu81GV4y3ZY4/CtwlVLqwl7KfQdj1HMNxgjx2R7KPgk4lFK2tgNKqTOB0/znUErdhjECvQ+YDNyKsVnkQ+EYr5QaClyL4e5tP4whfMv817wHuB2jQ4Ou6/uAt4Cvd6rua8A7uq7v849qXgGmAdcDU/32PquUuth/7WuB72EI/XhgAYZYCUIycQy4XCmV18353wJPYwzIhvpf/ww43622KKWGYPy2c4FF/nI/xhjMdEAplYvxuy7EGLAd6cHmu/x2n4Ph6bkHQ4Pa+A1Gx+IWjKmmOuCbAdcyYejLP3Rd39+5ct3gpP/twxj3pyTg85kYuvKI/30ecCXwJ13X64LU59Z1vambtmRh3LP5GB2mN4AVSqkJ/rqHYHQynwGmALOB+zE6KG33wgHcjKFji4BPurnWP4Gz/f+/GuO7XNO5kFKqGPgIo+N2PsY9/AJYrZQqDChqAv4HuBeYxACEbMQNse6lyavnFwGjEYwfz2bA5H/fxVvT6bMz/GWG+9+PoqMnKQdjhHV9wGf+AGwIeH8AuLNTvRf468nt4do6huepEWNUpGO4+If20t5/BXYHvL8WaMI/avPb3Aws9b+f779Odqd6/gq8FFDnLsAS6+9TXvKK1Qtjc9uDgBfDy/MIxh9QFVDmUWB1CHV11pafY3hXgnqYOeWNKMXwHr0I2DqVOUBXT9KKTmVeB57x/9+O4VH5aqcyn3DKo1Lkt/PfQmiTDagGfhJw7Bt+fcn3vz/bX9+1IdTXrt09lNkK/LDTPR3VTdkHgHcDv69O539GRw/aKAL03n9sPgGeJP9nPulUj8Lw7t8T8N3pwPmxfoZj8RJPUmLxPYxe/G3BTvpXLryhlDqslGrAGCEAjAxWXjdGUK/gH5n5Aw5vAJ7wvy/0f/Z3SqnGthenvDDjerH3h8B0DC/PPIxgwbeVUvYAm7/uD6A87q/7V53sXYExOlzmf38zRserLW7gLMAKlHeysW20BUbgogU46A+Ev8U/QhSEpEHX9Y8xpsTOx/iNF2NMs6zwe2S7JQRtmQWs0bv3ooDhjVgLbAeW6LreGoLZnWN7yv12g6E/Vrp6U9YGmu7/t9ed3P32PAl81e+BAsOL/YKu69Xh1tcZpVShUupBpdROpdRJv05N4dQ93IbhXdqulHpRKfVtpVRpQBV/w/DQ7VFKPaSUuk4pZQ3Xjk6cBczqpJ0NGB2s8Z3KbujntRIS6SQlELquH8RY/fELpVRG4Dml1AhgFcbo6waMoMRF/tM9/ZCeAC71u12vwPDUtLnR256Pb2N0dtpe0zB+QJ/2YvJxXdf36Lq+W9f1D4CvYEzZXe+3eSlGMPc/MVzYMzBiECwBbfZgxAS0Tbl9DXhc13VXgI11neyb7r/OFf46yjE6l1/BcCv/GPiikwAJwqBH13WPrutrdF2/T9f1qzEGXAsxvMNBCUNbeus4+ICVGLEtk0M02dXpvU7Xv1s9XbcSI85mSojXexhjKvEypdQMjM7fIwHnd2O0I9T6Ankco4P6Xf+/0zE6gVZoX1hzBXARRofkOmCXUmqh//wWYDTGtKcLw7O0xR+Y3ldMwDt01c+JGF6mNrwhdmoHHdJJSjx+hfG9/Uen42dhBDHeo+v6x7quf8GpEVdPvIHhYl6GMa//qq7rVQC6rh/HWPkx0d/Z6fwK90fTNree7v/3AmCzruu/03W9TNf13RgjmM78HzBNKXUnRgft0YBzGzE6drYg9h1qK6TrulPX9dd1Xf8uxmgsHVgcpv2CMNj43P9vkf9fF8Zq1EBC0ZYyYG6glzgYuq7fiTEIe8/fCekPe/z2zu50vH0ZvK7rPow4q5uUUqM7V6AMsgPK7wQ+wBiU3QHs0nV9dcD5GgxP+r8Efi6gPksP9+AC4EFd11fouv4pRqzVmMACusF6Xdf/W9f1CzBW5d0ecL5R1/UXdV2/G6OzehqGl76vbMTo8JUH0c+orDZMNKSTlGDout6A4Qm5t9Op3RgjqnuVUqOVUosxlqf2Vp8HQ0TuAK7CH7AdwA+Bu/05RqYqpSYqpRYrpR4OwdxspdQQpdRQvyA+jBFP9Ib//BfA6Uqpq5VSY5VS38aIQeps4yGMWIQHMOIlAtMgvIux+u8FpdQ1SqkxSqlZ/vwnXwdQSn3VP603TSk1ErgJyMRYxScISYFS6n2l1J1KqTOVUiP9CxsexFjV9J6/2H5gklJqilKqQCmVSmja8iDG35OXlVJz/eUWKqWu6GyH/w/8E8A7/oUifcI/tfcwhmd9oVJqglLqlxgdh0Dv0g/9bfhEKXWHXwdGK6WuweiEdF4M8zCGFt6EMUDrzDcxlvyXKaWWKaUmK6XGKSNn3Ua6TlO18QVGZ+10pdR0jBjT9g6pUmqOUurHSqlzlFIj/N/PGfh1Sin170qpm/zfzWgMz7gXI96yr/zJb8NLSqnzlZFw+Dyl1C+VUnP6Ue/gIdZBUfLq+UWQ4D8MMdpK1xQA38Lw/LRgxAxc7i8z339+FJ0C+fzHp/mPVwPWIDYsxpjnbwbqMVzEP+nF7s7LTysxVrQEBhFaMASpxl/v0xi5SPQg9V3tr+fGIOfSgF9jCLwLI4D0deAi//m2VXW1/jZsp1Owp7zkNdhfGDGNH2JMObcCh4CngMkBZfIwptbq6JgCoEdt8ZeZgBGQXef/nW0FrvSfu42AFAD+Y/+N0UE71//+AF0Dt3/U6TMdAss5lQKg3l/Xgxgrwj7t9Dk78FP/b7/FrwXr/O1K61Q2FajCCAov7OZeFmKs+N3lv5cnMDpcN2PkEIKuKQBO9+tQi79t38QY4D3uPz/Ff+8r/Nc+CPwvfk3GCCIv87e1LYXA1QH1/4wwA7f9x0YC/8DQ6LbrPoU/ZUKw7y6ZXsp/EwQhrlFKfRMjXmm4ruvOWNsjCEJ8opR6F6jVdf26WNsiJD6ScVuIa/wB6uMwghX/JB0kQRDaUEqdjpFzaC1GAPQtGNNnV8bSLmHwIDFJQrzzJ4yss59jJI4TBEFoQwf+H8bU01qMlWHX6LouyWKFiCDTbYIgCIIgCEEQT5IgCIIgCEIQpJMkCIIgCIIQhKgEbr+w4xm9qamn7PSJj91uZ7C3EaSdg4lotvGWs7/W47YWiUQy6BfIMz+YkHb2j5KMkVw4eUFQDYtKJ6nZ00yzpzkaVccNyqMGfRtB2jmYSIY2RoJk0C9IjuchGdoI0s7+4va5uz0n022CIAiCIAhBkE6SIAiCIAhCEKSTJAiCIAiCEATJuC0IcYTdlIHVlBqVum3KhjXF1q86XD4nTb7GCFkkCMJgIpr6BbHRMOkkCUKcYDdlkGHNhCitE7NYrPi8vn7VYdWt4EI6SoIgdCDa+gWx0TCZbhOEOMFqSo2qwEQERVRHioIgJCYJoV8QtoZJJ0kQBEEQBCEIMt0mCEI7mz7ewmO//Rs+r49LrrmY625fHGuTBEEQQibSGiaeJEEQAPB6fTzym8f48R9/wB+e/z0fvf4xh/cdibVZgiAIIRENDZNOkiAIAOzevoehJUMYUlKMxZLCeZfNYf3qDbE2SxAEISSioWEy3SYICcjyZ1ZTWeftcrww28ySG+f3qc6ayhoKhuS3v88vymfX9t19NVEQBCEo0dAviI6GSSdJEBKQyjovzWOu7Hp836o+16nrepdjSiXCchVBEBKJaOgXREfDZLpNEATAGHVVVVS3v68+UU1eYW4MLRIEQQidaGiYdJIEQQBg/JSxHDt8jOPlJ3C7PXz0xhrOmndmrM0SBEEIiWhomEy3CYIAgDnFzNf/4yv857d+ic/n4+JFFzJibGmszRIEQQiJaGiYdJIEQWhn1nkzmXXezFibIQiC0CcirWHSSRKEBKQw2xw0yLEw2xwDawRBEEInkfSr106Sw+GYCPwz4NAY4Ceapt0fNasEQeiR/iyTTSZEvwQh/kgk/eq1k6Rp2hfAdACHw2EGyoEXo2yXIAhCvxH9EgShP4S7uu1iYK+maQejYYwgCEIUEf0SBCEswo1JugF4JhqGCOFzsvIka1avpcXTTFpKOnPmzyanMCfWZglCvCL6FUeIfgmJQMidJIfDYQUWAd/v5vwdwB0AU24Yx9ixYyNiYLxiNpuw2+0xu37tiVpWvPoK+RfYsVtT8LpaWPHqK9yw9HpyiyKXADDW7Rwo4qGdNmXDYrFGrX6zSZFqs/W7HpPZhF1PrGdC9KsrsXzmRb8iSzy0M9r6BdHTsOysrG7LqmBpvIPhcDiuBr6ladqlvZV9autjelNTU0j1Jip2u51YtnHVc6/hndGE2XpqNYDX5cW82c6VS6+I2HVi3c6BIh7amZuSjzWKIpNqs+Fsbe2xzB9/9iAbP9xEdl42f3juvqBlXG4XtZ7qDse+MefuuN6/RPSrK7F85kW/Iks8tDPa+gW9a1go+gVdNWx01jgunXplUA0LJybpRsRVHTe0eJo7CAyA2WqmxdMSI4uEwcBFV83nJ3/6QazNiAaiX3GE6JcQDaKhXyF1khwORzqwAHgholcX+kxaSjpeV8ddlL0uL2kpaTGySBgMTJk1mczsjFibEVFEv+IP0S8hGkRDv0KKSdI0rRnIj+iVhX4xZ/5sXlrxEvnzMjBbzXhdXqrfb2TxokuClk+WIMlkaWcbuq6zWlvJfMfCfu92PVgR/Yo/wtUvSI7fdjK0MZBE0C/Z4DZBySnMYfGixZg322n5WMe82c7iRYuD/qBOVp7kpRUv4Z3RRNpcE94ZTby04iVOVp6MgeXRI1naGcieTdvJ3fMRezdvj7UpghAy4egXJMdvOxna2JlE0C/pJCUwOYU5XLn0Cq678VquXHpFtwKzZvXa9hEbGHP/+fMyWLN67UCaG3WSpZ1t6LpOxcfvcvccO8c+epdQF2EIQjwQqn5Bcvy2k6GNgSSKfsnebUlAi6eZtDCDJNvcvm5cWLAmhNu3L+1MZPZs2s4ludUoZeHinGo+27ydcTNPj7VZghBxwv1ti37FP4miX+JJSgLCDZIMdPumziZh3L7JFAzaNgo7p8QY55xbmhKR0dh937+f7932I44ePMrXLr+Tt196NxLmCkK/COe3LfoV/ySSfoknKQkIN0iyJ7dvJHOYRJq+BIMmKoGjMAClVERGY/f+6p5ImSgIESOc37boV/yTSPolnaQkoC1I0lg10UJaip3Fiy7p1v2cqG7fcNuZyBzbvYeTzcV8vOvUihBd12ndvScuXdaC0B/C+W2LfsU/iaRf0klKEtqCJEPBcPt2zYablhL/6f3DaWcic8EN18TaBEEYUEL9bYt+xT+JpF/SSRK6EOj2NdlMCeX2TbY8I4IgdET0S4gkErgtdCEwh4lzreo1h0m8kIx5RgRB6IjolxBJxJOU5HQ3cmlz+8bDxomhkqgBm4Ig9A3RLyHaiCcpiRlsIxfZNFMQkgfRL2EgEE9SEjPYRi6JHLAZL1RVVPHAT/5MbdVJTCbFgmsv4aplV8baLEHoguiX0Jlo6Jd4kpKYwTZymTN/NtXvN7YnZGsL2Jwzf3aMLUscTGYzt/3rLfzphd/zmyd+yWvaGxzedyTWZglCF0S/hM5EQ7+kk5TEDLYMr+Fumil0Ja8wl7GnjQEgzZ5GyejhVJ+oibFVgtAV0S+hM9HQL5luSxKCBTj2lOE1XvY+CndJbLLkGQGoPlHD2++8TZO7Ebslg0suvoT8oryI1X/i6An2f7GfCVPHRaxOQegrnbVgyulTeO/9d+Nav4LZ3ZMtol/xp1/iSUoCugtwBIKOXIC42PtosAVmRpLqEzU8+fzfaD2jButsL61nGO8j5fVpaW7lN9+5j6/cexvpGekRqVMQ+kowLXjv43e5cO5Fcatf3dktGpZY+qVC2VDO4XDkAI8CUwEd+IqmaWu7K//U1sf0RFl22VcSaWnpqudewzuja0CgebM96KglsLzJZMLn8/VYPl7s7g/x8H3mpuRjtVhDKvvPZzRaz6jpcm9s2/K4/kZH0M+k2mw4W1t7rdvj9vDLb/+G6XOmcfXNC7ucd7ld1HqqOxz7xpy7VZeCcYLoV3Di4ZkPlXC0IF70K1y7+0M8fJfR1i8ITcN60y/oqmGjs8Zx6dQrg2pYqNNtDwCva5q2xOFwWAEZWiYQ4e5lFG757tzJ/c0em6h7MA0ETe5GrEHuTZO7sV/16rrOn//rIUpGD+9WYBIQ0a8EJxwt6ItuiIZh4ssFAAAgAElEQVQNLImkX712khwORxZwAXAbgKZpLsAVkasLA0K4S0vDKd/mTs6fl0Ga1YzXZbiTL5x7Ee99/G6X4+EEIsqS2O6xWzJodXUdidkt2f2q9/MtX7D61Q8YOW4E/3rDvwNw87/cyKzzZvar3lgh+jU4CEcLwtUN0bCBJ5H0q9fpNofDMR14BPgMmAaUAd/WNK1b314yuKvjwb0ZKoEi0DHAMfiP/WTlSZY/vxzbLBMmq8Ln0mkt87HkuiVdyq967jUaR9Vw/NNqdJ8PZTJRfHo+x9+oY9QNRf1yM4drd3+Ih+8zHHd125x+3gX29ntT80ETt153e7fBj6FOt/VGIk23iX51Tzw886ESjhaEo18wODQsHr7LaOsXRE/D+jvdlgLMBO7SNG2dw+F4APge8OPAQg6H4w7gDoApN4xj7NixfTQ/MTCbTdjtiTEasNvtLLthGe+/9SHN7ibSLVksu2ERuUW5Qcu7mlzgU9RubUA3gfJBmi+D9PT0Lm2ub6rn6IZjFM/PxWw14XX5OLL6GD6PwmKzdChrsplw4g75voVrd3+Ih+/TpmxYQhSZYSOG8fWbvsHrr79Jk7sBuyWT6266lPyigm4/YzYpUm22fttpMpuw64nx7CP61S3x8MyHSjhaEI5+weDQsHj4LqOtXxA9DcvOyuq2bCidpCPAEU3T1vnfL8cQmQ5omvYIxogtKUZi8dBzDwer3cqCxRd3ONad/W+veofiS7MxW/M6BD6+veqdLiOoyiPVFF2Tg8lqQgdMVhNF83LY/1gl7lZ3l1GYhfDuWzh294d4+D6tKTZ8Xl/I5TOyMljiuLbDsZ5GWZEchTV5EubZF/3qhnh45sMhVC0IR79gcGhYPHyX0dYviJ6G1Znruy3bawoATdMqgMMOh2Oi/9DFGK5rYYA4WXmSVc+9xvPPPM+q516L+vLRFk8z9eWNlD36GRv/tp2yRz+jvrwxaLBhwfA8dCfQNm2r6+hOGDFxuGSPFWKO6FfsiWf9AtEwoWdCXd12F/AP/8qQfcDt0TNJCKS7oMJoZmJtrXWxb/dhRiwtwmwz4W31se/Fw4wp6JqUKzsjG1e6oqW5BZ/uw6RMZKbbseZlMWf+bP/KkBbSUuwsXnSJZI8VYoHoV4yId/0C0TChZ0LqJGmatgU4M8q2CEGIxSaOlSeqKL2mELPNcDSabSZKry6k8sWqLmXnzJ/dHiSpLOB1+aha28CS6y6jrrqevV/sx212YvGmMuX0KSIwPeDyObHqVojLEGg/umFnIiH6FTviXb9ANCxSJIR+QdgaJtuSxDmxyLPhS/WQmm7H6/Vh5N5TpKZb8VmDX9Pn9lGzpQmUDrrC5k7jyN5yVr7zCiXXFGBOTcPr9PLM809z43XLGDlpRNRsT2SafI3gAqspNSr1m8wmXO7+rX53+ZyGnYIQAomgXyAaFgmirV8QGw2TTlKcE4s8GxZvKl6XD3OqGYUhM16nF4uv68O/ZvVahl6R18W+Fx9cweivFmJO9Y8gU82ULC5gxfKV3PWjb0bN9kSnydcYtU6IXbcnUsC1MAiId/0C0bBIEk39gthomHSS4pxQNqENJRvswZ2HWLF8ZbvbeNGShYycNCJoHYuWLOSZ55+mZHEBKalmvE4vR16q4sYly7qUr62rJdfacddts9UMad52cWk/nmrGbXL2O4utIAiJQaT0C4JrWHZ+Vr/0a8782d16u7rTsFa9mVXPvSb6lSTIBrdxTk5hTq+b0Pa2ceLBnYd45vmnyb3GytClOeReY+WZ559m+yc7gtaRnZ/Fjdcto/ZFFxXL66l90cWN1y0jOz+rS/mKqgoajzd3uJ7X5YUWQ5w6HHd6UU6zbPgoCElCJPQLgmvY08/9g3/87R/90q+XVrwETlP7CrY2utMwd5OH5roW0a8kQjpJCUBOYQ5XLr2C6268liuXXkFOYU6PAZGdWbF8JSWLC7q4jV/UVvRaR2A+9jWr15IzN43GpkZqa2ppbGpk1MIh7H/laJdlstc4FnHkpap2kWkbzRUPKQrZbkEQEp/+6hcE17Bhi/Koaajtl37lzE1DmVTQpf7BNOzA8gomLhkp+pVEyHRbghJOQKTb7MSc2mlKLNUMNm+Hefi2OiqOVbBt11ZKrikgJTUNjz9gsTCrCOtYHymZJkxKge6juaGZIcOLjZFip2WymTmZhnvc5MTiS+XGJcvYuHkDZqupyzWTfcNHQUgmwg3oDq5hJpS141KqcPWroaGBFK/O4kWLgy7176xhpUNGkDUsM2S7hcRHOkkJSjgBkRZvKl5nx/l1r9MLrUaMQOc6yg9UBA1Y3Pnng0y6qBSUX5iUQqVCfWUjX/7GrV2uO3LSiC4Bjjs+3SEbPgpCkhNuQHdwDfOhuzruPdoX/aoqr2n3dnWms4ateu410a8kQ6bbEpQ582eHnA120ZKFQae+rnEsClpHWp41aMBiSpqJY+/U0FLVirPeSUtVK8feqSFvSOhBi3Pmz+bYazXseeMQe948yJ43DnHstRrJYisISUQ4+gXBNezoihryMnNFv4SoEhVPUmNdI7pJx2SSPli0aAuIDCUb7MhJI7jxumVdpr5GThpBydjhXer4+8NPB/c8Oc34nD5qtjQY3Wsf+Jw+bFlpXa7ZEyaLibzpmSgL6G5oLQt9vx9BEBKfcPQLgmvYsqU3BaxuE/0SooPSdb33UmHyi3uW6p8erSY949TD19DqJGdUMcrv6rTnZjB25qk08UopLKmWLnXFK/GwoWB39HeJfdtKkrYltB6/52lo5nD0s1tIzU8xXNa6jrPaQ+rWbLLzskK63qrnXsM7o6u72rzZHrUMvKEQz99npIhmG78x5+54z7MbMrLBbWwZaP3K2lXERVfMD+maol+xJVrtHJ01jkunXhlUw6LiSfredXO7NMTp8nDi5KkkUzsOV7Hp2dXt72sbWmi2WjGbDe9Ts9NN3rih7Z2q0skjyBuW115evFTBicReSYGjNm+KG7PH0h50bS1Ko7GhsX2Po5y8bHadOETGxaNDul4sMvAKgpAYDLR+5Rfl0fh5a8jXFP1KPgYscDvVmkJp0akHrrQoh8tnBd9wEKDF6aa8sg4AHZ0Vq7fyeYux30ptQyt6RjrKZIwGdLuNHH8HymxNYdK5k5K2ExWpvZLaAhYDe+47Pt2B19dEdm52e7ma47XYhlhDvl4sMvAKgpAYDLR+eV1eqsprGHVDUUjXFP1KPuJ2dVtaqoVxJQXt7+8tKey27KHjtdQ3Gx2o4ycbefuR11BK4XJ5OKkUqTYruq6TWpxNZkEWAMWjhpA/LD+6jYgBkRrptGW39aS4SfFYWLRkYdDsucfermbMFSUhX6+nDLyCICQ3A61f1e83kjckJ2gqlGDXFP1KPuK2kxQOI4pz2/8/Fbh4RnAP1Y79FbS6PAC8+85mjrQYG+VVNrZizTJGAl6TYugZI0EpcgqyGTKyOLrGR5hIjHTa5/Q75Rm58bplXYItxwwdhzWja7ba7q4XbsCmIAjJw0Dr1+JFl7Bm9dqQryn6lXwMik5SqEwZPaT9/7MmlgQtU13XxM5DJwDYtKacLW9sAqCioYW0HDs+n07u2CEUlhbjdLYyctIIzCnmoHXFgjnzZ/P035+m2dKIDx8mTKS7M7hgzjz++IsHu+zdFowVy1dSdGEWx9fUous6SimK5mexYvlKbvnGMn8pI+D/zHNn8dbbb2KbZeqw2mPJdd2PrLrLSSIIQnLTnX4tu2VZt/tPdiYc/Wq75vLnl4esYaJfyUVSdZJCIT/bztzTRwO0/xuIruus2X4QX1UttXWNrH5/OxZLCiebWvHZ0zCZFLmji8grKcBqsw64J6quup6TjXUUL8jGZAGfG469VsOLr7zI6GVDMKem4Q0YWQUTmRZ3Ey2bvRRfmIvZasLr8nH8vVp8raYuAY5vvf0mznonzVs8oHTQFTZ3eEtqBUEQILh+HX+rjp2bd7F6w7uUXFPQq4aFo18vrXiJC+dehM/to2ZLk2iY0IWQOkkOh+MA0AB4AY+maWdG06h4RinF3NNHtQcELppzWpcyH28/QM3B4xyqaWDj65tQJkVVQzOWbDu6DqPPmYAtI42M7HSy8rIiat+K5SsZcV0hyqxjVTouXVF8SQ612xu7ZKBdsXxll4zYAM4GDyOW5LVvH2K2miiel8vuvxwl/7aOQZW2WSaat3gYd9kpofK6vGEHWgpCtBD9ShyC6deI6wpZ+edXmfSt0pA0LBz9yp+XwYpnV3YI3AbRMOEU4XiSLtQ0rSpqlgwi5k4d1f7/o9X1PL5mI6S6yDWbWXb2DMr2VODx+fjs2Em+0MHpdNNsMZNqSyW3JJ+hk0pIy0gjI9uYEw/VzQzGHkcmSyomnw8UmHUdk9WEMnfa4yjVjNvkDFrHiInD8TY3GSKjAB28zT4yC+1dAhyVBWP0FVi3LIkV4g/Rrz7Q35xF0H/98llMmDNU0CzawTQsHP0yW406Qg3cFpIPmW6LIker6/neW6+TNS+DFGsKR12t/OS9t/j1gssZlt/Rg+RyGwHl63aVs3fDF+w+XkeLycSx4zWs3b2DnOkZ5E3NwpyueOrZp7j5hpuDCo3Fm4qn2U1amn8UpYxss7q30x5HTi8WX2pQuwvzCmmxp1BfU4+ue1HKTHZmNk0Wvcteb7ob0Dt2wGRJrCAkPpHIWRQYRB3KVH8w/XI1u/E26kGzaAfTsJqtVfhqPDQ7nYAPMJHiNJFqzgi6V6XFlxr0uGiYAKHv3aYDbzocjjKHw3FHNA0aTDy+ZqO/g2T8+FKsZrLmZRiepU5YLSlYLSmcP2Ukt110Br+88Xx+d/1cmrzVTL2rlBEX5tN0uJn6zxpRVh+P/+px3vrDy6x9/iPWrfiE+pp6muqbOWvaVI4+X0FDjYvmBi8NNS4q36im5aCTlqoW/55FLRx6vpJFSxYGtXvO/NnUftSMcnvJtJpQbi+1HzWzaMnC9v2Wmmob8Lq8tJb5sLWmhbwHkyDEANGvPtBTzqJQWbF8JSWLCzCnmvF5vB2myYIRTL+OPl/BOTNncOj5ypA07JwJoxiTlc2oi/KYvLCQURflkWeyUWwr4POHDrD/tcPsemkv+187zM6HD3Dl1ZeHtY+ckFyE6kmaq2naUYfDUQS85XA4dmqa9kFgAb/43AHw8yuKGTtubIRNjS9MJhN2e88jjQblwWqzdjhmtZlpUJ5eP9tGk8VDTk4GPp+PIbOzMZnNmEwmTmoneeobV1LT0EzlySZWvbuNk61udnyyjcZqF/VPVGBOM5GaY8HqNWPGTM2GJjDr4FXkZ+cxtHRIUDtcTS6c9a04tzbQalF43Tq0ZDK0dAjLbljGy39/kdbyg1AygVtvuRWA99/6kGZ3E+mWLJbdsIjcotwu9cYzZnPv32eikwxt7Iaw9IuCZoqKEyv1R18wmRQ+X/fbUh3YcRBLa9fjnr1VfJqzNaRr1ByrxrMuDd2ng+4DZUKZFPXHWvj0LaOOCbMnkVdsJAPWG5pJ9Vmp+qQBlaLQPTqpPivp6ORn51Gzob5XDTt8vI6D9VW49rRiMiu8Xh2by8afl83FmmLmt6+8w+HKg5QWFXDmWaNYt2k/k3In8tkzO3F6naSaU5k8ZRKHtxzkMAdDvJunOOPi6aRnpof9uf6QLL/taLUzO6v72OCQOkmaph31/3vC4XC8CJwNfNCpzCPAIwCeD+8f9HsfhbKHTKaeQkNra7snCcDj8lKg20Lef8buTsHT6gEzpJjA4/Xhc/mwu1NwOluxW03YizL55qXTjPpVPRPPHIKryYPXo3N4ay3VVU007m2iqDgN3Q3pQ9PIGm3nrVff5sqlV7Rv/dLGW6veofB8K6XZee17HB2uU7y16h2uXHI5E2nlp1fm859bW7GkW1BKsWDxxR3qSLTvPxn2PkqGNgYjXP3at/ynenNz84DbOdCkp6fTUzu9e3M5PtaJ2XJqwsHr9lHclMqdo/O6/VwgGzLtqHGpYIIUk47Hp8AH2btM3Dk6j5ONLfzfq+uYc/0FANRadMbdPqHL1Ndnzx7zB1cXdTj+9qp3ugRXW07LY0hBE0Mzs9tjkrZvaOThtz7i/tsXMS3Vy5NX5XHfZi+3L5jOZdUNxgfPCJ4WJhy27T3Guk92MHnO5H7XFQ7J8tuOVjvrzPXdnuu1k+RwOOyASdO0Bv//LwX+K3LmDV5um3NmQEySGY/LS/37jfxgwXkh1/GfVy3gq88/z/DFeWAz43N5KH+phseuui5o+VpvCynWlPaO2eRLhnLkZB3HPjCTZjdRV99Ca7kH15EmqstP8M6RVmxZdryWFPJGFjJi6kiqKivIKXax/YNGvIAZKJqcQVVlBYe27uKyvDqUsnBpXh3rtu5m5PQJ6LrO+hfe4+xrL+zS6RKEWNEX/RpRnEtTk7WnIoMC4w9O9+28+7Lz+N5br2PvpF93X3Zxl5jK7viN40q++oqhX1abGVer19Avx3WUFuWQmZ4KX1S0l+8u47bb5MTV6ObopqO4W51YbKkMm1kIQYKrGxpryBkFB3ac0i97Ghw6Ucv6HXuZW9CAUinMKWig7PMDnD3VmPXQdZ0nX/2IW790Xp81rLK2EdyR3zReiB2heJKKgRcdDkdb+ac1TXs9qlYNEoblZ/HrBZfz+JqN1HpbKTLb+MGC80IWGIDpY4dxhcpixzMVnEw1ken0cUV6AdPHDgtaPtecxlFXR+9Va5WLhnon+RflYE9JQ/coKt8/yfzJI/jVDQsAqKhp4ERtI6+88gnlHx3ks3VOMselYx+eSordzBcb68mu1an75BPOnWY8NrNLU3jzk7Xo08ZzaOsuhu/fxKGtJYycPqEfd00QIoroVx+JhX51zrh9fFMVJzZW07qvlU0Hq7GPsoGC1jonm/5yguEpeWyrX8VRl4fL71oEQOvRRnY5GxlycV57nqQjr1Ry5NMTaK4P+MsVRt2zS83cV1bGWVPGoJRi/Y69OI9sZ8OOYe0dJ0HotZOkado+YNoA2DIoGZafxQ+uuqjPn1+/Yy/Xj/cye8SpvevWHPKwYce+oD/kYN6rqvfqKPlSAZh1zCYdjxnyZ2WhdhhudF3XeWPNFm790nmcMXYoTd56tk2pwZJnpvGwE2etG1edl7rD9VTWVXFfhZn6Vi9fnpPLpXkePtmyi/p1n/CdyV5+6+80tY3ExMMkxBLRr/4x0Po1Z/5slj+3nGZvE64aJ0PHZzK1tID8oWnsmdmIKVuRYtbxeBW+Op0zduTxy6XnccWPHufo/mMMGz2UwgmlpE44QUpaCkoplFnHPsqO7zMfz687SGlqDsV2xVfPyWFOQQMbduzjrClj2FS2idsme3k8oOMEkfEwCYmLpACIc3bvL8fXlMvWz04d03UwHygPKjLBRn+zRpfQnKVTV1+PMkGKD7IzsnBbfQBdRlBuq5eRBTlUNTWTU5CGqSCd06cP5fhzxzm/IJtd5SdpbDjG91c2YbKmULHqBSZnNPFxi85In4dD/ik4QDxMgpDE9KZfSimOl1ex99P9ABzetJcx9RnkZGegj1bk6jZuu/RMfv/hhxRmpAfVsPU79nLlkGYOPf8BW81WjtVVkFGUSt3RZnSMsCRvpc6w0jRumjeSw5UNHK6p4EcfKEoKcjEfKEdHZ0ZGFXhamW6v6tCJC8fDpJSifF85qZkDm7HbZrPR2hokyn6QEa122kbYjY1fgyCdpDjn5qvmhf2ZzqO//17xLtsaKijJAIsJ3D440dBMrikLXdfZVLaJe+ektbuec81ptPhaGZqdiclkxufz4nF5mTlyJLcsvJCHnnyO3y0Yzn2b0/nGLUt4+O/LWVZyhJ0n3FRWNvHhX57ni4kTSclKp273Tn46N4WnOnmYBEEY/PSmX1npqdx9znhaqmoB+PJ5EykpPKdLuVxTGocb6rpoWI4pi01lm/jePDv3bW7kf29dyt2Pvkx5UzN2SypmZcKr+9BHWpiRX8wtiy7ioSef4955xdy3OY1brr8GgIeefI5bRzgpsps4Qzl50q+FQBd97EnDTh87hJsaW/D52zNQpNpsOJOgkxStdg7J7X4Bg3SSkoBpBfmsf28bvstz0K0mfG4fre+dYPqsqe2BjDV1rcwp8LJhx74OU3b1rU1k2WztAeedAx+fWvURMzKqyE3VWTDWwuxSndOGmTBPGoOOzp76Laz7wkv90ZM8+8O/UTxmGMWnlTJ03DDyihMrTYAgCJGlbZun3uhOwzKLCrro1/evvShAv1rb9esrC87qol8bduxr9yJlWnwoFJkWX7s3SUfvUr4nb1KK2cxFM8dF8A6Fhqxu6x/u/OHdnpNOUhJQU3WSUdVeKp+poMWiSHPrjPJZqK6q5eDundw5yUNrUwvTClL4S1kZZ926lF8vuJxfvfwWDdWHyc4fwa+vvpyheZm8vPJ17p1xKvDx2Ve384WvlX2VOgodHfisxslE6xEaa49z79l2lFLous59m3XuXDqHNzftpey1Dex1ealudmLLzaB02miKRxWTkZMR25slCELc0Z2G7a7ZxV2XW/qsX/eVlWHPKWLXF87uNWxG8EBvITmQTtIA0N/Av7b932q9LeSa07htzplhrTCZMGY408x7GJfegFl341UWdjdnslVXzC1ooLm5mVd2urjqtGbmFKS0BzJONnv4z+k+/nnUzdC8zA6jMDBGgYvGw6GTVm4/O4PH1p3ka+fksPawl60+U5eybSOxy2aN49KZY3ny1Y+4acl8PD4fr2zYzbYNX9CEiVqnm5zh+UyYO5msvEzMKeaemicIQhSJtX5BcA3bWm3l0Elfv/RrTkEDW/UhOKZlcW6pOWQNO2vKGAnmThKkkzQA9GdpabD937731utB93/rjt37yzlRnsLL1a143G5SLF4y8nMpb9xHVUYKzxxrIc/m5Rfvt1A0JI8i3QhkPM1yDIvuYpK1gg079gUNwtx3pIV6p2LP+wpLs7M9GHLb4X1Q2n3AZts92bTTuCdL505mqb9cq8vNgWO1vPPeVj6uqMVnMmEuyGL45BEMGzcMm90W1j0UBKHv9Ee//vrmZh5bu4608amYzAqfV+fVP+3iJwsv5rJZ40OuJ5iGNeqKeqfOlsN91y9dh21H9kFJLq+83xCyhunoki4gSZBOUpQJFhgdzsijp/3fQl2ae9PCC/jLExVkZJq495x07lvnxWOz8T93LOWhJ59jdH4K956T2n78xivP4+G/P8f1xU6GZ5lw+1r5Z9lG7rzVEdR2Xde7BEP21Mbe7onNamHSyCImjSxqL19d18w7nx5k88ef0WQ247ZZyCzIZtL5k0mzp2Eyh7oNoSAIodJf/fq/DzYz/LbhmC0BGbTP9vLrJz4Kq5MUTMPyMotRSpFSf7Bf+tXWzlA1rL1sH++JkFjIX5Yoc8rFe8pV24au6zyx8kN0vfsMrUYG7Y7TTSlWM7Xe1pDrWL9jL3meSuYO97uMh0Gep5KnXv242+OnWY6RlwYeH+Sl0T4aC3bNntoY7j0JhlKKghw7158/mV/fPI8/3nge/zV/Cl8ZkcvxFz7m/T+8zNt/foVNb5RRe+Jkj/dCEITQ6a9++Uxe0gvSSM22tr/SC9Jw4wm5jjY7OmtVY2U56S1H+61fvbUznHsiDD6kkxRF2kZh55aeCvwrKyvr0LkwXLbd/8hyzWl4/LtTt+Fxeck120KuY9e+I2ypcDM230pli5lxBVa2VLjZ8vmeoMe3fr6Xl3Y08fetbh5c7+TvW928vL2RL/Yd6XLN3toY7j0JlaLcDE4bWcwPrp3Nw1+9hD9dP5evleRSt2o9H/z+Rd77y6uUvb6Biv0VvVcmCEIXIqFfFjfs+uc+9iw/0P7a9c995KWnhVwHBNewTyuclB1x9ku/QmlnOPdEGHyYf/azn0W8Ut+hT37mdrsjXm88YbVa6a2N63fsZYx7F6U5xg9KKQXuVg65shlWmMsrr7/Dv50Nz2+q4sxpk4O6bMcW5PPqu59hKbVgMpva90/6zvwLyEizhlRHq9vDzLQKJgy1k5ZmIz09jZx0C6pgPAuGNTJ+SDqPrqtj/mkF5KRbOeTN5eqSWi4cncLFY1IYmWOm0G6i2jaCM8aP6HBNrzmVse7dQds4vKjrJpg93ZNg5UMlxWwmP9vOeaeVsnDmWBaePoISn49DW/dR9uYm9m87QMWRSux5maQFiWkK5ftMdKLZxjNLz/nPqFQcA5JBv6D35yES+nX5jPFsLj/CqAX5lE7NoXhsBhkVJu53fClk/YLgGtboUlwwxsbkksw+69eZ0yaHpUnR0q/+kgz6BdFrpy+9kPThpwXVMIlJiiI9ZZsNNf9GT/snrdu+J6Q6ug1Y9AcmPrXqOFluF3etamHKqGI+33WAeruZDw86KUhXVDXrnHCmotv3sX7U0A7XfHXddkoyQs8IHm4G8b6ilGLyqGImjypuP7bz4AleeHU9W2oaITON7DFDGDZ+GENGFvdQkyAkJ/GiX93ZsuV4E7vqTLywu6XP+tVTQHcwTRoo/RLiBxUNN6Hnw/v1wZ7Yqj9JrdoD/2Y0B+QQSufOW5eGHAAYiToAfD4f3/3F/Tx0mc6dbyj+50f3oJRqrzvFnILH6+mQXbu/14wHdF1n484jrN17nD1V9ThtqQyZPoqho4eSU5gda/OiQjQTzn1jzt2J9QD0QDLoF/T9eRD9ij8kmWT/cOefRsFZ1wZ9CGS6rY/0x+0XCZdtb3W05TaZNn5EjwLw5MoPmZd5kGwr5KV6ee8QON1uxrh3UZJt4g+rKzh7lB3laeWt3U2cbT8Wd67mvqCUYnhhNudMGMYV00dz3awJuPcdZdf6XWxYvY2Dnx1C2Syk2qxYUi2xNjciyHRbaCSDfkHfnwfRr/hDptv6h0y3xRmRcNn2VkcouU18Ph+fbt7IbfNNNLt8zB5uYvnqDeCdCS1t03DOdjd2Wz6RwehqtqSYuXjWOA1SQ4kAACAASURBVC6eZWwpUN/UyupPD/DxB9s51uIioyiH0+adTmFJQYwtFYTYIvolJBPSSYoBfdm0Npw62lZg3DbZy+MBeTw6Z879+6sfsXS8m2a3ojhDcbxJZ8l4N7vMZm52LOa7P/89/35eKj/8qJmbHYsxmZJnMWSW3caicyex6NxJAByprGP5+9v4pKGVWpebMedMZNQZo0lNS42xpYIwsMRKv9rOtWmY6JcwEEgnaRCyfsdeZmRUgae1faPGYKOzbTv3saM5eIDj33Wdq8a48emKhaPdPLXqY25deH6smxYzSgqzueeqswBwe7y8tmkvq//6JvVuL/bSAibPP4PsMLdaEAShK93pV9u5Ng0T/RIGgpA7SQ6HwwxsBMo1TVsYPZOE/tA2Crt1hJMiu4kzlJMny8o4c/LoLplzf/udLwcNnrzj5uv4j1/cz9K5xgjt7GGw8uP1+K6cK6MxjKm5RWdPYNHZE/D5fOwpr0Z7aQ3b3T5aTIpxcyZRMrGUFIuMQeIF0a/EoDv9OmvKGIAOGva/994aNBBb9EuIJOE8Md8GPo+WIYOZ7rLKhpptNpw62kZhmRYfCsi0+Jhur+KpVz9mbkEDNXVN7UtfAzPHwqkNHH/1t1e4aoybTKsCFJnWU6MxoSMmk4kJpYX8aOlc/rjsfO676kxK9x9n7YMreeeR19hTthu3c/AHVCYAol99JB70K1Cv2jTsqVUfiX4JUSekoa7D4SgBvgT8Evi3qFo0COkuCDGcjSNDrWPXvnK++MLJvkodhY4O7KhuxWf+lFsuNNPa1MK0ghT+UlZGRm4RenPXQMbPd++n2uJjf62v/fjWE2DO2Qvisu6RbLuNm+ZN4aZ5U6hrauWdbQdY/9jrVLq9ZI0ZwunzzyA9Mz3WZiYVol/9I9b69VmNk4nWwzTWnuDOSZ52DXv2ve3oI4vZ+tmp1W+iX0KkCXU+4H7gu0BmFG0ZlHS3QWQ4G0f2FIjduY4JY4YzzbyHc0vN/HH1ce6aX8xj6xsZnQstLS0UZ5g43tzCnIIUnCNncfbU+V2uN3HqHtIOvs/sESmYTWa8Pi9rDnlwjhSBCYdsu41rZ0/i2tmT0HWdzw+e4Ol/vMfRFhcZI4uYfP7UQZuTKc4Q/eoj8aBfaw972eozM7egoYOG3XB6Js6R07t00ES/hEjSayfJ4XAsBE5omlbmcDjm91DuDuAOgJ9fUczYcYN7SaXJZMJut/dabs2WXVxQ3ESK2cL5RU18uvcos6dN6PZ4d3XMzKwGTyszMqt7rONAeSXelkKeee0YdpeLb7/hxOO1sPW4k43pbgrSTVQ2udEzUxliOcGF55zR5XptdezYBaAAHV2HlKPByw8GQv0++8NZUzLaYyt2HjzOs29uZnNNI5kjijjt/CkUlRZF9fpmc/TbGG+IfnVPKM98POjX6aOHsuPwAWozUvmgsaZXDRP9GrxEq516dveD1V4zbjscjl8BtwAewAZkAS9omnZzd59Jhoy1oWT+7C6rbDiZX9vquHVEOUPsOhVNiicPDe+xDl3XO2Sh/c0Pv80jTz3fp0yzksk1+uwtr+Klsn18fqIO29A8xs6exNBRQyJ+nWTMuC361T29PQ/xol+ds2iHo2GiX4OLWGTc7tWTpGna94HvA/hHYt/pSWCEU5wKjDZuc1tg4amAw47HN+zYx1lTxnTIZdQxkFF1CcQOVsfn+8tZOt6NSVlYMt7Nr//2ClcNDV5WkqjFnrHDC7h3uJGk8kBFDSs/+ZzXX1xLwcThjD97ItkFMiXXV0S/+k5f9OvsqWM75DKKhH49tepjJgbsuRbsmoIQLWSNchTpdmPZHjK/6ughBTK2WPZCadc61L4jfLqljG9dZny1l4xJ4a8vfM5m+4QuAY6SaTb+GDUkj3+50tgiYfOucl55+RM+qWsmZ8wQpl06E1u6JK8UBoa+6FfnfGyR0K/lb5zKoi3ZsoWBJqxOkqZpq4HVUbFkEBJuZlpd1/nLExq5plY2btzYIZBx9ohTX1VbEGIwcXjilQ/aR2EAJqX4ynQTuzKLuEVWdiQUMyYMZ8aE4QBs31fB3/72JpUuD6UzxzJmxjjsWbJKLhxEv8KjL5m1dV2nbGNZu4bNnDWz3/rVlkX71huu7XtjBKGPiCcpjli/Yy95nkoam1vJM1WyYce+sPdJ2rZzH/u9qbxx6JTXyKebaErZJ8tfE5ipY4Zw3xgjTum9bft59e9vU6VMTDx/CiUTS7FY5acsxJ7OGvbuuu2UZIh+CYmLKGuc0DYCS2lu5t7ZVu5b18zGjRv5f1929BpcHch9/35b9IwU4oILzxjNhWeMptXl5sVPdvHWaxuxDS/gjEtnSkoBIWYE07CcLBe3XB+6hol+CfGG5GiPE9pGYHOH+4MSh0Gex/AmRZtwMucK8YPNauHGC6bw2Dcu48dzJnD4hY946w8vs2vDLsnyLQw4sdIw0S8hmkgnKU7Yte8IWyrcjM23UtliZlyBlS0VbnbtPxL1a58KtIx+h0yIPEophuZn8csbL+Chm+dxem0d6x96lfeeeJuK/RWxNk9IEmKlYaJfQjSRTlIf0HWdR198N6IjlwljSlg2I4v8/Fzy8nLIz8/lxhlZTBhd0qst/RlFtWW9vWdOGmVlZTIaS3AsKWauOXcSf7z9Yv5r3mQ8H37Km/e/xKa3Nsl3K7QTLxom+iXEOxKT1AfW79hL04HNbNhRELHlp+EGaAfaEur+Sd19vi3/iOQdGVwU52XyncXnous6727Zz8rf/JNmu42Zi86W3EtJTrxomOiXEO9IJylM2kYu351j53962bMoHPq63DbU/ZN6/PwMMwCzS819qkeIb5RSXDxjDIvOO519hyv4vxWf8FFVPRMuns74WeNjbZ4wwMSLhol+CYmATLeFyamRy6mMr4lqS+DngbhokxBdivMy+dGSOfzt6wsYU17Fe/e9wMfaB7Q0tcbaNGGAiBcNE/0SEgHpJIVB28jl3NJTI5dYzYNHwpbd+8vZ0pDLQ59ltr+2NuSy+0B5tMwW4oQUs5mb5k/l4a8v4K4pJex84m1W3vcC5fuOxto0IYrEi4aJfgmJgky3hUF3exn1dR48cI+jttFQsGPRsqUvU3zC4GNCaSG/uXkeza0uHnl7K6++uJYJF01j9NRRpFhEIgYTkdSw7rQqFA0T/RISBVHAMAgMTExJScHj8fRr/6BgQYuhBjL2NdBbELoj3WblnoVn4XJ7WLV+Fy+9sYn800cx7ZLpWFOtsTZPiACR1LDutCoUDRP9EhIF6SSFQeDIxW6309TU1Oe6ggUtAiEHMsooSogWVksKi+dOZvHcyWzcVc5DD7xMxoThzLz8TKw26SwlMpHSsO6CrkMNxhb9EhIFiUmKEcGCFuMloFIQ2jhzwnAevfNyvjwin08eXMlH2gc4W5yxNkuIMd1plWiYMNiQTlIM6C5osWxjWcwDKgUhGDPGDePBry3gGxOHUvbIa7z/1Lu0NktnKRnpTr98Pl9cBIULQiSRTlIMCLZ0dbq9inxPpSxnFeKa00cP4Q+3X8w900ex+dHXee+JtyV9QJLR3dL7p1Z9JEvyhUGHxCTFgGBBi/uOtFDvVLg+y2w/JoGMQrwycUQhD9x2EQeO1fD7R1+HBd+NtUnCANFd0PW2I/ugRIKx4wFd19lxsJLtB6pibUpESbXZcLZGflA28jQbl58V/FyvnSSHw2EDPgBS/eWXa5r200gamGxI0KIwWBg1NI8Hbr841mZ0i+hX5BH9ij+cLg+vrP2CsgMnafaaqWiCnJLxFE9ZEmvTIkp6WhrNLS0RrzetsKjbc6F4kpzARZqmNTocDgvwkcPheE3TtE8iZaDQO6HmTxIEoQOiX3GA6Fdk2b6vgiNVDazeWUVVs4/U3KHkTJpPwaWjGJ6dy2DdbKi/q8q7Iyc/vdtzvXaSNE3TgUb/W4v/JZF4A0x/N4IUhGRE9Cs+EP3qHxt3Hmbj3ip2V7ZQ60kjY9g4skeexfDFYxibbo9a50EIMSbJ4XCYgTJgHPBnTdPWRdUqoQP93QhSEJIZ0a/YIvoVHidqG/n84HFe33aCqmYfKfYcrEMnUXjaAiZdNibW5iUdKpzlmQ6HIwd4EbhL07Ttnc7dAdwB8PMrimeNHTe4Rwsmkwmfzzcg11qzZRcpe99kdqmFNYfc+MZfyuxpEwbk2gPZzliSDO2MZhtTzr8n7v/qiX51ZKCeedGv7tF1nW17j/LBjmMcrm6mokknJbuYvPFnMWba2VisqSHVYzaZ8fq8UbY29kSrnZOHZjFv0pCgGhZWJwnA4XD8FGjSNO233ZXxfHi/PthdfwPl3tR1nYeefI57ZzS3Z7S9b3M6d966dEBGY8nixk2GdkazjdmX/zDuO0kg+hXIQDzzol8dqTzZSFVdEyvWH+RgdQs+WzbWkikUj5lK0Yi+d8zjrZ3RIlrtnFCYzqJzJgZ9IENZ3VYIuDVNO+lwONKAS4DfRNhGoRsivamuICQTol+xJVn160jlSTxeH1v2VbHjyEmcLg8nnBb0tDwyho5h6AVfYkZ+9yuqhPghlJikocAT/nl9E6BpmrYyumYJbchGkILQL0S/Yshg1q8TtY189NlRDlQ2oOuwt7KVVHsmHq8XT8YwbBnZ2AvOZfhVMwEotci+h4lIKKvbtgEzBsAWIQiSk0QQ+o7oV2wZbPrlcnt44eNdrN5ZiV4wjqKxcyi88DQAptnSMZnNMbZQiDSScVsQhLDxeL24PUZA7J6jNZwXY3sEYSD47hNryD73eqZ9eQop4hlKCqSTJAhCBwIXc2zaXcHuo7XowOaDJyElDYCqJjepOUMAsGTkSidJSAqsaRmUTBLHZDIhnSRBSEJanG7eKtuHT9fx+nTW7K0lxZKKDlQ0eLBl5gFgLx5BwThj25HSmYXY0u0AgzajryAIQiDSSRKEQcZn+49zuKoBgC+O1XGswfAMNbl8tPz/9u41Rs6rvuP4d6579168vt/jS2LjJg42ITQQkhTSpElLlRenUEEpFLkvQCqFConQKioSokUVhaoI4SaRi6BBpyUgwi1ALgQIDokdkzg48XWx1/bevPf13Ofpi1mvd/Z5xp7ZndnZfeb3kaJkz8w8c0529qf/nOc85wkvIRgIkAWW77qLaF1uZmjz7g1Te7KoABIRyVGRJLJIxJMpfnP0LADpdIafvDZAMBSemv2pa24DINK+mraNewBo2NXKdavXA7Wzl4rItYxMxNn/s9foHS9to8necUdfImqMiiSRKstms7x8/DyZbBbHgSd/e55kNrev2cWxBIHmTggESKUztG5/R27BaBDWPbCDaH1uJkg3KxC5uv7hcX5yqItfHR8k1bCMTe/8Kzat2VDSMTZVqG+ycKlIEqmQWCLFsbP9Uz8/c+QCFydSAPSNpYhMrvtJpTI0bNxFXWMLACvv/guaW9sBKC3CRWS6WCLJU4dO8sTBbsbqVrBm959w8+07q90tWURUJImU6EzvELFErtg52z/Gz1/vIwCTp72yNLS0AhBLZVmyeTfBYBCAzt33sH7VWgDWV6PjIjXAcRyOnO7h4Z8dI9W0nKZNu9n+wY9O/R2KlEJFkgi5qfj+wWEAnKzDd184zUQydyPFvpEYNHQQCOR2C042LKWxYxUA4YZNbDa3T92HSsWPSHWcHxhl/1O/49RIgMYNN7H1vf/E0mUrtA5P5kRFkviS4zgMjEyQzeau7Boai/Gjl89OPd43EiMWapkqblJ17YRbOqceX7X7w3R0LgNgaSBAKByZx96LSDEGhif4xrNHOdqXhLZ1bLr1I7x1jb6qSPmoSJJFY3QiTvbyRocO/OjgaUYu5U57pTNZjvbGqW/ILWROpNKkm1cRrc/t60OokU3v+BihUO4jvzYcnlr0DLryS2SxGLuU4HsHTnDg1DCxSDvb3v0Rdi9bNfWFR6ScVCRJVaTSGTLZK5ffHj7Zx+vdQ1M/9wxP0J+IEpwMvkQqRSy6lLqGRiA3U9R5/btYsnzt1GtubO3QugMRn4knUyRSGZ44cJJXu0foc9rZcMsfs+Nt2wlHNMMrlaUiScoqm83yvV8fJ5HOTLWdGZigLx7i8vc8B+ifcGhou3J6q2HpGlbsuG/q50gkwg0dy+ap1yKlmYgn+cFvTjDtDi4AnOgZYziWYffGVs+ZjZ0bl/GmDfpcF/Lsb7voHb5E73CMM6MOsWSKRH0n4UiUVTe/h7VvWc2WxuZqd1NqiIok8XSmZ4jDp/qor68nHo8D8MaFMS7G84O/fyxJsHnp1M+OAyt2vp3GaQVQdFsD25atzHvdtgr2XWQuPrH/AKlkcurnC2Npos3tec9xCLBmz31E6+rz2uu3N7OhoYmTA72ex/7VkQPEfnHM1Z5Jp2lIDdPSWOd6LJ1K8PatHVOzqtNtXNnKjdetdLUvZF3nB3mlq590JsuvTg4RjkS5MOZQ19xK24YdtK3fSnhzlOtXrql2V0VUJPlR//A4L5/ocbX/vn+ck4MpAlwJ22QqzWC6Pm99DkCooYXlO++lobGRWCwGQOPGFjYvX533vM0V6L9INW1//2fz1qfN5jO+fJ33toOF2q8mGY9x+PwZz8eeeuMw8effcLVnHYfQpQHam+tdj6VSSe68YSktTY3EE4m8x1a0NXLTltWu18zGS290MzgW58WTg4yng/SPJQk1LyXU1MryHfdCIMCWPZsIR6JsKcs7ipSfiqQq6+4f5vSFwYKP/+L1AcZT3gsSe8dSrm+4ANlQHZ1/8M68YgigblsjmzeUFkda0CxSXdH6BtZcd73nY4XaryadSvLrk6/T0NBALBDLe2z05DFSB466XuMAzvhFlra4Z7oymTTr2iKcG80yeilBur6DQDBI/cotNK/cRPudq1jesUy7wsuidM0iyRizDvg6sBLIAvustV+udMcqbXgsxvHugVm99tDpi/ROQHLalLyXdCZDXzxEQ1Phc+iZaAttW24t+HjnOzewaan3GgZtkS9ydX7Nr7kIR6JsuOFG7y9AN9xY8vEymTS9Z05x3SadRBf/KWYmKQ180lp7yBjTAhw0xvzUWvu7Qi/42/2HSSZTZetkJSSyQdq37oFZXDbasm0VW950U1EzLDodJVJVJeeXlCYUCrNaBZL41DWLJGvtBeDC5H+PGWOOAmuAgiGz5/0P6hSNiFTdbPJLROSyktYkGWM2AjcDL1SkN+J7T//gcYYnEq72tqY67rrvgQV7bFn8lF8yV8qv2lN0kWSMaQa+DXzcWjvq8fheYC/A+rs/xObN/j7RFAqGaGpqqnY3Kq7c4xyPZ8jecLe7/fjTc36fuRy7Fn6ftTDGQpRfbrXweVB++UulxtnauqTgY0UVScaYCLmA+aa19nGv51hr9wH7AL76zDHH76fbauWqr3KPM51J5+20Pb19ru8zl2PXwu+zFsboRfnlrRY+D8ovf6nUOEdGHHLXdrhd8x4OxpgA8Ahw1Fr7xfJ2TUSkcpRfIjIXxcwk3QZ8AHjVGHN4su1Ba+0PK9ctEZGyUH6JyKwVc3XbLwHdXllKUmgR4omjR2jtfLOrPdPtvaNwKXq6zxDqdG/MOXL0CI/bx1ztWhDpf8ovmS2vDDv2u9doXrKDSDR/J3Pll39px22piOGJBKlt73K1p15+kYljz7vao2l3QVWqbDpB3OPYqXjMsy/Dx3425/cUEX/yyrBg3zgjLz1BQ2f+feWUX/6lIknmVVP7Mlbder+rPVKGP/jVG7d6hsmFUfd97EREStW643Ymjj3PmhkZpvzyLxVJ4jJ9mjkcCpPOpAEYPHeajjX5N0Pp6T5DNp1g9catee3nu46zzOMPfrTvPInn3BcYJbpe9uxLKVPK57uOkxyMu9pjQ/1FvV5EFr9S8utq7V4ZNvTqM6SGe+makWHKL/9SkSQu06eZs8Hg1GWp410P0zIjNEKdg8SPPe/6BpTu6vI8tlPXSNOb/9TVHu89Necp5TQRGm+8x9Ueu3Ci6GOIyOJWSn5drd0rw5xsliVvM4QaW/PalV/+pSJJijbUc5bYM/+b15bFIT3YXfQx0qMDDD79iKs9k7jE8KB70aLXgsh/+8dPkAg1uNoTwwN0Tgy73zNR/HqBcu16q91zRRYWr/wCiPecZVWRx7jU/Rqp0X4I5O+eU0p+gXeGJRMJoi98l5add+S1l5JfUJ7sUX5doSJJihaob6LlLX+e1+Zk0gw9+6jrudFw0PM8fbCpnfY7P+xqH/jBv+PUu3c9Tabdm6slQg20vvujnscI1TW73zMU8uxLW1Odq63QgvNSF0mW6zgiUh5e+QWQ6D/t+XyvDAtGGmi/62+YecFkKfkF3hmWmRhi4uhzrgwrJb+gPNmj/LpCRZJUxMq163nAvM/V/srhQwQCXldkl+Eq7QCEwu6PdEtrm2dfREQK8cqwXH557cFcjl0mAkDAlWHKr+pSkVTDCk2pvvLrn1N3bsjVnomNudrGjzyFk4xz7sD389qTvz+M1/0fMrExnMmFlPkcUqlkXks2Ps7w4EW+9IXP57Wn4zGP11fW+a7j2qtEZAEpR37l2sdd+QXeGeZkHcZ++yTNO/9o5iOu/AIYGuh15RfMf4b1vvQk2cGzrgxTfl2biqQaVmhK1Tn4GxrfdKerPd5zgsylkby2bDLGkrcZGhvzbzqY7DnpeezgoYMQdH8TyybGGX/1qby21OA5Iis2U//W/Cny4e97313CyaS59MqPXe1RUp7PL0WaiKafRRaQcuQXQLCukaZtf+hq98qwtsbriZ8+5Mowr/wCyKYS1L/9r919L5Bhqf7fuzKsLPmVTtG05wFSHR35/VB+XZOKpCpYtIvinKzrqo7M+CAEAiST+eNJpQr8YQcC3tPVWYfU0Ln8t0vGAO9z+l6C4ahr/xKAi08/qlkgkTLxU34BkM268gsKZFggAIGgO8M88uvysYsVbGgmGK13ZZjyq7pUJFXBQl8UFwiFvcMkGGLkp1/Ja8okk4Sb2gmEo3ntBaMhnWDwqf9yHzpaT/vtH8w/Rnyc8deecT03Gxtz9QMgGB/xXOCYTRf//7utqc6zPRq+5r2gizpOocWWIouFn/ILIBAMEGjudLV7ZlgmTfLCcQYvns0/tEd+AQz86D88++iVYZl0CmKjrgwrJb/AO3uCoz2EgsWvm1J+XaEiSYoWDIX5zD9/Lq/tcw99xlUgXU20sYWl797rar/wxBcJhGZ8HAOB3D8zdKzawMc/9emi3/Nx+1jRE9aFvpmVcoyrHUdEqsMrvyCXYcUKR+uoW72VtpvvzWv3zC/wzC8oLcPKkT2P28dItbXP6Ri1SkWSzEkmncLJeizEdoqfZq6Wnu4zU9PY03fm1TS2SO1YrBmm/JofKpJqWKEpVWJjjL34XVdzIDnhassmLjH+ivsY6fFhz1NfdZkY8V/udx/byeJkZnxfcrKkek+5nt8ccdx9noVkOuu5M28pp+FqcfpZZCEoR35BaRkWO3qETDxGfKI3/9he+QWQSXnmXTkyTPk1P1Qk1bBC3zZ6us8Quil3iWsgEMBxcn/QGcd9CW2kqZWWXe6t9LP9J7339iiw38eXvvB5IpEZp+0iHbR1LC3p1Fql6JuZyMJSjvyCWWSYB8/8Ato7Vyi/FjkVSVWw0Kv6bDpB/NjzAAQDAbKTIRNNu68ACaXjTBx6wrO9FM0Rh/EKfeMq9P87XIZLa0VqjZ/yC8qTYcov/7pmkWSMeRS4H+iz1u6sfJf8b6FX9as3bp2axg1Nm8b1On227aY9nldeeD33aj7y9w/OoqfFKddibFmclGHl5af8gvJkmPLLv4qZSdoP/Cfw9cp2RSTfot2PRRaa/SjDpAqUYYvfNYska+1zxpiN89AXWYQqOfVe6f1Ypvd95tUh4h/KMLmaxZphyq/5oTVJMieL+dvQ9L43NTUxMeF99YuI+NdizTDl1/woW5FkjNkL7AVYf/eH2Lx5c7kOvSCFgiGampqu/cRFaFlbC0MnnwXyrw5pb2uZ1zGHQ2GyHvd5C4fCZe+Hn3+fl9XCGGer1vIL/Pt5WCj5BfOXYX79Xc5UqXG2ti4p+FjZiiRr7T5gH8BXnznm+L2q9XPl/o67r9w7aOY453PM6Ux6atHlzPZy98PPv8/LamGMs1Vr+QX+/TwslPyC+cswv/4uZ6rUOEdGHGCl52Ol3ZBKREREpEYUswXAY8AdQKcxpht4yFr7SKU7JrLQ92ORxUEZJtWiDFv8irm6rbgtR0XKbLEuqJSFRRkm1aIMW/x0uk1ERETEg4okEREREQ8qkkREREQ8qEgSERER8aAiSURERMSDiiQRERERDyqSRERERDyoSBIRERHxoCJJRERExIOKJBEREREPKpJEREREPKhIEhEREfGgIklERETEg4okEREREQ8qkkREREQ8qEgSERER8RAu5knGmHuALwMh4GFr7b9UtFciImWi/BKR2brmTJIxJgR8BbgX2AG8zxizo9IdExGZK+WXiMxFMafbbgFOWGtPWWuTwLeA91S2WyIiZaH8EpFZK+Z02xrg7LSfu4G3Xu0Fnc111DnJufRrwWupgTGCxukntTBGD8qvAmrh81ALYwSNc64ao4Xni4opkgIebc7MBmPMXmDv5I/fsNZ+oKjeLVLGmL3W2n3V7kelaZz+UQtj9KD8KqAWPg+1MEbQOCupmNNt3cC6aT+vBc7PfJK1dp+1do+1dg+wvUz9W8j2XvspvqBx+kctjHEm5VdhtfB5qIUxgsZZMcXMJL0IbDXGbALOAe8F/rKivRIRKQ/ll4jM2jVnkqy1aeBjwJPA0VyTfa3SHRMRmSvll4jMRVH7JFlrfwj8sITj+v7cKLUxRtA4/aQWxuii/CqoFsZZC2MEjbNiAo7jWsMoIiIiUvN0WxIRERERD0WdbiuWMeZR4H6gz1q7s5zHXiiMMeuArwMrgSywz1r75er2qryMMfXAc0Aduc/I/1lrWsKtfAAAAmhJREFUH6purypnclfml4Bz1tr7q92fSjDGdAFjQAZIT17FJdMov/yjljJM+VVZ5Z5J2g/cU+ZjLjRp4JPW2u3ArcBHfXibgwRwl7X2JmAXcI8x5tYq96mS/o7col6/u9Nau0sFUkH7UX75RS1lmPKrgspaJFlrnwMGy3nMhcZae8Fae2jyv8fIfTjXVLdX5WWtday145M/Rib/8eXiNWPMWuA+4OFq90WqS/nlH7WSYcqvyivr6bZaY4zZCNwMvFDlrpTd5BTuQWAL8BVrre/GOOlLwKeAlmp3pMIc4CfGGAf4Wi3szitX5+f8gprJMOVXhWnh9iwZY5qBbwMft9aOVrs/5WatzVhrd5HbofgWY4zv1mgYYy6vPzlY7b7Mg9ustW8G7iV3iuX2andIqsfv+QX+zzDl1/xQkTQLxpgIuYD5prX28Wr3p5KstcPAs/hzrcZtwJ9NLgr8FnCXMeYb1e1SZVhrz0/+uw/4DnBLdXsk1VJL+QW+zjDl1zzQ6bYSGWMCwCPAUWvtF6vdn0owxiwDUtbaYWNMA/Au4F+r3K2ys9Z+Gvg0gDHmDuAfrLXvr2qnKsAY0wQErbVjk/99N/DZKndLqqAW8gtqI8OUX/Oj3FsAPAbcAXQaY7qBh6y1j5TzPRaA24APAK8aYw5Ptj04uauvX6wC/nvynH6Q3K0cvl/lPsnsrQC+Y4yB3N/8/1hrf1zdLi08yi9fUYb5R1XzSztui4iIiHjQmiQRERERDyqSRERERDyoSBIRERHxoCJJRERExIOKJBEREREPKpJEREREPKhIEhEREfGgIklERETEw/8DlxkTPI09eM0AAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# 我们画出决策边界\n", + "from mlxtend.plotting import plot_decision_regions\n", + "import matplotlib.gridspec as gridspec\n", + "import itertools\n", + "\n", + "gs = gridspec.GridSpec(2, 2)\n", + "fig = plt.figure(figsize=(10,8))\n", + "for clf, lab, grd in zip([clf1, clf2, clf3, sclf], \n", + " ['KNN', \n", + " 'Random Forest', \n", + " 'Naive Bayes',\n", + " 'StackingCVClassifier'],\n", + " itertools.product([0, 1], repeat=2)):\n", + " clf.fit(X, y)\n", + " ax = plt.subplot(gs[grd[0], grd[1]])\n", + " fig = plot_decision_regions(X=X, y=y, clf=clf)\n", + " plt.title(lab)\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "使用第一层所有基分类器所产生的类别概率值作为meta-classfier的输入。需要在StackingClassifier 中增加一个参数设置:use_probas = True。\n", + "\n", + "另外,还有一个参数设置average_probas = True,那么这些基分类器所产出的概率值将按照列被平均,否则会拼接。\n", + "\n", + "例如:\n", + "\n", + "基分类器1:predictions=[0.2,0.2,0.7]\n", + "\n", + "基分类器2:predictions=[0.4,0.3,0.8]\n", + "\n", + "基分类器3:predictions=[0.1,0.4,0.6]\n", + "\n", + "1)若use_probas = True,average_probas = True,\n", + "\n", + " 则产生的meta-feature 为:[0.233, 0.3, 0.7]\n", + "\n", + "2)若use_probas = True,average_probas = False,\n", + "\n", + " 则产生的meta-feature 为:[0.2,0.2,0.7,0.4,0.3,0.8,0.1,0.4,0.6]" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3-fold cross validation:\n", + "\n", + "Accuracy: 0.91 (+/- 0.01) [KNN]\n", + "Accuracy: 0.95 (+/- 0.01) [Random Forest]\n", + "Accuracy: 0.91 (+/- 0.02) [Naive Bayes]\n", + "Accuracy: 0.95 (+/- 0.02) [StackingClassifier]\n" + ] + } + ], + "source": [ + "# 2.使用概率作为元特征\n", + "clf1 = KNeighborsClassifier(n_neighbors=1)\n", + "clf2 = RandomForestClassifier(random_state=1)\n", + "clf3 = GaussianNB()\n", + "lr = LogisticRegression()\n", + "\n", + "sclf = StackingCVClassifier(classifiers=[clf1, clf2, clf3],\n", + " use_probas=True, # \n", + " meta_classifier=lr,\n", + " random_state=42)\n", + "\n", + "print('3-fold cross validation:\\n')\n", + "\n", + "for clf, label in zip([clf1, clf2, clf3, sclf], \n", + " ['KNN', \n", + " 'Random Forest', \n", + " 'Naive Bayes',\n", + " 'StackingClassifier']):\n", + "\n", + " scores = cross_val_score(clf, X, y, \n", + " cv=3, scoring='accuracy')\n", + " print(\"Accuracy: %0.2f (+/- %0.2f) [%s]\" \n", + " % (scores.mean(), scores.std(), label))" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.947 +/- 0.03 {'kneighborsclassifier__n_neighbors': 1, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "0.933 +/- 0.02 {'kneighborsclassifier__n_neighbors': 1, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 50}\n", + "0.940 +/- 0.02 {'kneighborsclassifier__n_neighbors': 1, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 10}\n", + "0.940 +/- 0.02 {'kneighborsclassifier__n_neighbors': 1, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 50}\n", + "0.953 +/- 0.02 {'kneighborsclassifier__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 50}\n", + "0.953 +/- 0.02 {'kneighborsclassifier__n_neighbors': 5, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier__n_neighbors': 5, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 50}\n", + "Best parameters: {'kneighborsclassifier__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "Accuracy: 0.95\n" + ] + } + ], + "source": [ + "# 3. 堆叠5折CV分类与网格搜索(结合网格搜索调参优化)\n", + "from sklearn.linear_model import LogisticRegression\n", + "from sklearn.neighbors import KNeighborsClassifier\n", + "from sklearn.naive_bayes import GaussianNB \n", + "from sklearn.ensemble import RandomForestClassifier\n", + "from sklearn.model_selection import GridSearchCV\n", + "from mlxtend.classifier import StackingCVClassifier\n", + "\n", + "# Initializing models\n", + "\n", + "clf1 = KNeighborsClassifier(n_neighbors=1)\n", + "clf2 = RandomForestClassifier(random_state=RANDOM_SEED)\n", + "clf3 = GaussianNB()\n", + "lr = LogisticRegression()\n", + "\n", + "sclf = StackingCVClassifier(classifiers=[clf1, clf2, clf3], \n", + " meta_classifier=lr,\n", + " random_state=42)\n", + "\n", + "params = {'kneighborsclassifier__n_neighbors': [1, 5],\n", + " 'randomforestclassifier__n_estimators': [10, 50],\n", + " 'meta_classifier__C': [0.1, 10.0]}\n", + "\n", + "grid = GridSearchCV(estimator=sclf, \n", + " param_grid=params, \n", + " cv=5,\n", + " refit=True)\n", + "grid.fit(X, y)\n", + "\n", + "cv_keys = ('mean_test_score', 'std_test_score', 'params')\n", + "\n", + "for r, _ in enumerate(grid.cv_results_['mean_test_score']):\n", + " print(\"%0.3f +/- %0.2f %r\"\n", + " % (grid.cv_results_[cv_keys[0]][r],\n", + " grid.cv_results_[cv_keys[1]][r] / 2.0,\n", + " grid.cv_results_[cv_keys[2]][r]))\n", + "\n", + "print('Best parameters: %s' % grid.best_params_)\n", + "print('Accuracy: %.2f' % grid.best_score_)" + ] + }, + { + "cell_type": "code", + "execution_count": 45, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.940 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "0.940 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 50}\n", + "0.940 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 10}\n", + "0.940 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 50}\n", + "0.960 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 50}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 50}\n", + "0.960 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 50}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 1, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 50}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 50}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 10}\n", + "0.953 +/- 0.02 {'kneighborsclassifier-1__n_neighbors': 5, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 10.0, 'randomforestclassifier__n_estimators': 50}\n", + "Best parameters: {'kneighborsclassifier-1__n_neighbors': 1, 'kneighborsclassifier-2__n_neighbors': 5, 'meta_classifier__C': 0.1, 'randomforestclassifier__n_estimators': 10}\n", + "Accuracy: 0.96\n" + ] + } + ], + "source": [ + "# 如果我们打算多次使用回归算法,我们要做的就是在参数网格中添加一个附加的数字后缀,如下所示:\n", + "from sklearn.model_selection import GridSearchCV\n", + "\n", + "# Initializing models\n", + "\n", + "clf1 = KNeighborsClassifier(n_neighbors=1)\n", + "clf2 = RandomForestClassifier(random_state=RANDOM_SEED)\n", + "clf3 = GaussianNB()\n", + "lr = LogisticRegression()\n", + "\n", + "sclf = StackingCVClassifier(classifiers=[clf1, clf1, clf2, clf3], \n", + " meta_classifier=lr,\n", + " random_state=RANDOM_SEED)\n", + "\n", + "params = {'kneighborsclassifier-1__n_neighbors': [1, 5],\n", + " 'kneighborsclassifier-2__n_neighbors': [1, 5],\n", + " 'randomforestclassifier__n_estimators': [10, 50],\n", + " 'meta_classifier__C': [0.1, 10.0]}\n", + "\n", + "grid = GridSearchCV(estimator=sclf, \n", + " param_grid=params, \n", + " cv=5,\n", + " refit=True)\n", + "grid.fit(X, y)\n", + "\n", + "cv_keys = ('mean_test_score', 'std_test_score', 'params')\n", + "\n", + "for r, _ in enumerate(grid.cv_results_['mean_test_score']):\n", + " print(\"%0.3f +/- %0.2f %r\"\n", + " % (grid.cv_results_[cv_keys[0]][r],\n", + " grid.cv_results_[cv_keys[1]][r] / 2.0,\n", + " grid.cv_results_[cv_keys[2]][r]))\n", + "\n", + "print('Best parameters: %s' % grid.best_params_)\n", + "print('Accuracy: %.2f' % grid.best_score_)" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "StackingCVClassifier(classifiers=[Pipeline(steps=[('columnselector',\n", + " ColumnSelector(cols=(0, 2))),\n", + " ('logisticregression',\n", + " LogisticRegression())]),\n", + " Pipeline(steps=[('columnselector',\n", + " ColumnSelector(cols=(1, 2,\n", + " 3))),\n", + " ('logisticregression',\n", + " LogisticRegression())])],\n", + " meta_classifier=LogisticRegression(), random_state=42)" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 4.在不同特征子集上运行的分类器的堆叠\n", + "##不同的1级分类器可以适合训练数据集中的不同特征子集。以下示例说明了如何使用scikit-learn管道和ColumnSelector:\n", + "from sklearn.datasets import load_iris\n", + "from mlxtend.classifier import StackingCVClassifier\n", + "from mlxtend.feature_selection import ColumnSelector\n", + "from sklearn.pipeline import make_pipeline\n", + "from sklearn.linear_model import LogisticRegression\n", + "\n", + "iris = load_iris()\n", + "X = iris.data\n", + "y = iris.target\n", + "\n", + "pipe1 = make_pipeline(ColumnSelector(cols=(0, 2)), # 选择第0,2列\n", + " LogisticRegression())\n", + "pipe2 = make_pipeline(ColumnSelector(cols=(1, 2, 3)), # 选择第1,2,3列\n", + " LogisticRegression())\n", + "\n", + "sclf = StackingCVClassifier(classifiers=[pipe1, pipe2], \n", + " meta_classifier=LogisticRegression(),\n", + " random_state=42)\n", + "\n", + "sclf.fit(X, y)" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEaCAYAAADtxAsqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd3xT9f748dcnSdu0UFpq2RvZQ5QhIAioqIiKAziuy1dc3Ct63QMQZcgUB169Du518vMKBxe4EBFlOhBcDBFkyN6FlqYjyfn9cU7bNHSkI0nTvJ+PBw+ak5OTdz4Z7/OZRxmGgRBCCBEIW7gDEEIIETkkaQghhAiYJA0hhBABk6QhhBAiYJI0hBBCBEyShhBCiIBJ0qgilFKGUupv4Y4j0iilmltl17cKxDJSKeUOdxyhpJSaqJTaVsFjVJn3MJwqoyxDIWqThlLqTeuDaiilPEqpPUqpt5VSjcIUUgPgvTA9d0RQSm1TSk3027wbs+y+D31EkUcp5VZKjazEQz4N9CrD88t7GOGiNmlYVmJ+WJsCNwLnAAvCEYhhGAcMw8gK5nMopWKUUiqYz1FWSimbUspe3scbhuGxyi63MuOqSipaRsGQF5NhGBmGYRypyLGi4T2sVgzDiMp/wJvAUr9t/wQMoFYR238HsoCtwGOAw+d+B/AE8CeQDewFXvC5vybwvLU9E/gJuNbvOQzgb9bf7wBLioj5c2Cez+2LgdWAyzr2G8AZ/q/Rin8n4AVqFlMebYFPgQzr38dAK5/7RwJuYCCw0SqLH4CufsfpBiyxjnEY+ABo5nP/RGAbcJ1Vpm6gE9DVen2HrMeuBQb5PO4bq4x8/zW3/hlAX2u/vNua9Royge3ACL84W1hxZgF/AXdZz/HfUj43Z2KeWByzjv0rcIVfGfUB1lv3rwW6+TxeAf+xPisuK7ZpQFxFy6i0z6L1GShUhpXwvk0Etvns1xh4Hzji8/oeLst7aO1bF/PzfNB6j7YAt5by3lwP/GztvxN4Fqhh3XcGZo3meb/n2A/MLMd7o2H+FmQCHwG1gGutONMxWw2SivguPkDB78D7QKr/sf1eU4nf8bD8dobzycP6wv2SBtAQWG59GWr4vZG7gGswf2gGY/7IPOmzz1uYX+QRmD8qvYD7fT6IX1tfmL5AS2AUkANc5HMM36RxKeABGvncX8+K7TLr9oXWB++fQGugh/U8KwDl8xpPAh8CZwOd8Ul2PseOt17jV5g/Ht2sY20DYq19RmImnfVAf+As4BPrS5dg7dMB80dnEtDOer4FwB+A06c8M62y7gW0ARKBAcDN1jHaAFOsMmpjPS4F2IHZHFLf+men+KSxHfOL3QqYYZVda5/35GfM5pBzrbL5DDhBCUnDes6DmF/+vtZ7fRUw2K+MVgDnW2WwBPNHyGHtY7NeW08r1iFWGU7y+8yVuYwC+CzWscrh3rwyrIT3bSKFk8Yiq3zOtl7fBcANZXwP44HNmJ+1gZjfmUuA60t4b0YCx63X3RLoh5nQ5/rs0w/IBa60PgNfAN8BMWV8b05hnmCdhfldOGy9z58BXaz3/iBWMvL7Li6yyncAZtJZ5Hds37Is9Tselt/OcP94h+2Fm2+i2/qyZFJw5vO0zz4J1n3+Z3P/B6RZf7eyHjesmOcZgHnmk+S3/XXgI5/bvknDhnlW8ajP/Q9YH2C7dfsbYIbfMZtaxznb5zWmUUztwudxt1mv0/espx7m2c3/WbdHWsf2TXS1rfK73ef55vkdO8469tXW7YmYP6xNA3iPfgEe87m9DZjot09zik4aD/js47Di/Lt1+2JrH9+aVIoVZ0lJ40ngAD4nFX7355VRV59tvaxtbUs47v3AVp/b5Sqj0j6L1j5uYGQR34VyvW+c/kP3i/975Ld/IO/hbZjfmcZl+D7vBP7ht62fddzaPtsmYNaCnsH8brQo5bhFvTduCn9X/o15klfHZ9vzwI9+ZZxB4drHJVZ8rX2O7VuW31DKdzwc/xxEt+8xz9ycmGelFwOP+9zfEfOs532llOGz3Q44lVJ1MJsMwDzTKEoPIBbY69edEIt5pnEawzC8Sql3MM+aZlqbRwDvGIbh8TluL6XU3UUcojXmmTTAZsMwMoqJLU9HYJPh0zZtGMZBpdQW6z5f3/rsc1wptRnzTDUvplZKKf/nc1ox5TloGMZfvjtYZTkJ8+yqPuYPvRNoVkrsxcl7/RiG4VZKHcRMhFjxHjEMY5vPPses11uSbsAawzBOlbCPgfnDmWev9X89zKYLlFJ3ALdj/ljWwHyt/v2L5Smj0j6LxSn3+1aE2cCrSqnLMH/0PjUMY0UZ4+mG+XncE8jOVrk0A55VSj3te5f1fyvMpjwwE/+lmCdh1xuGscPvWIG8N3uNwv04B4ADhmEc9ttW1+9xmwzDOOFze7X1f3uK/i0I9DseUtGeNFw+PxwblFJtMM8abrW25X1YhmNW1f0dC+A5bJjNHj2KuC+nhMe9BTyslOqG2TZ9NmaC8z3uTGBuEY894PN3ST9wvowitqlitvvv4xvTXMzmIH9HS4npTcyzqEcwmzBcwDzM5Foe/mVrUPjLX9rrKk5pj/P6JHbf/W0ASqnhmJ+xMZhNPScxP19T/Y4TijLKU5H3rRDDMN5QSi0GBmE2TX2ulPrQMIyyDicvy/uT977ei9l84883+TTAbFrzWP/nK8N7499hbxSzraIDjQL9jodUtCcNfxOBjUqplwzD+JGCDt+WhmF8VtQDlFLrrT8voeghsz8CyZhtwxsCDcQwjI3Wsf8PM2n8bBjGr37H7eh7tlwBG4F/KKVS886glFL1ML9UT/vt2wtYZu2TjNkG/qpPTGcBfxpWXboM+gGPGIaxyDp2Dcy2ad8yy8Gs5VXUJqCOUqpVXvkppWpjvt51JTxuHXCHUqpGKbWNkvQDfjIM49m8DUqp5mV4bEllVNpnEYouw4q8b6cxDGM/ZoftG0qpz4B3lVKjDcM4Wczz+1sH3KqUahxIbcOqFe/GbAL8T3H7KaVswP/D/Lw/ByxQSn1tGMYqa5eKvDeBaK+UqmWVA8B51v+bi9m/Mr/jlSbah9wWYhjG75idu9Ot2xmYoyemKaXuVkq1VUp1VEpdr5Saae2zDXO000tKqb8ppc5USvVQSt1rHXYZZsfgB0qpa5RSLZVS3ZRS/7SqwiV5C7gBuAl42+++J4CrlFLPKaXOtp53kFLqNaVUfBlf+v8wO/PmK6W6WrWbeZhNK/N9iwh4SinVTynV2YrplPV4rLJqD/w/pdS5SqkWSqkLlFLPK6ValhLDFuAmpVRnpdTZwLuc/uOyA+ijlGqqlEq1fgTKYylmE9Lb1nvVBfNszk3JZ7gvYX5nFiql+liv7wqrKSZQW4DOSqmrrPfsXsxRN4E+ttgyCuCzCGYZXqCUaqiUSrW2VeR9K0Qp9aJSarD13B2t17Ybc0RR3vOX9h6+izkwY5FSaqAVz0VKqetKeOrHgHuUUuOVUp2s7+rVSqlX/fbpDNxkGMZHwCvAO9YJA1TsvQmEgfmZ66SU6odZq/nUMIwim6mp3O94pZGkcbqngIFKqYsADMN4ErMz7HbMH5pV1u2dPo+5BfNsewrmWcOHmCOtsM7chmAOYXwWc7jip8DlmKNqSvI/zFpKXQp+mLGO+zVm23ZnzPkmv2KePaVzelW5RIZhuDDPTrMxR2Ysx0wGgwzD8G3m8QLjrNf6I2ZV//K8s27DMDZjnj3VxByZsglzCGM8ZqdjSW7B/Dz+gDmEcTEF7dB5JgBJmF/uw5hNNWVmvSfXWK9xJeaJwufWcYudK2OdQffFLOPPMM9Yp1K4ia40r2ImqDcwh173xKzhBiKQMir2s2h5ELPPYAdmGVb0ffOnMPs1NmB+lmpgjvjLS8alvoeGYWRijkragHnyshnzB7bYH0rDMOZi9ktejlk+azHLdS+AUuo8zB/hW31qLw9Zr++/1u2KvDeB+AHz9+NLzHLeiPl+Fakyv+OVSRkVr42KKKDMWcT/NQyjWjZpKqUSMdu+xxuG8UK44xHVi1LqTczRYAPDHUtFVcsfACFKo5QagtkctRmzJjcBs/lAD2dcQlR1kjREtErAbK5ojtlMtQ5znsDBcAYlRFUnzVNCCCECJh3hQgghAhbpzVNSTRJCiPIp14rXkZ402LdvX7hDqBJSU1M5cqRCK1RXG1IWBaQsCkhZFGjYsGG5HyvNU0IIIQImSUMIIUTAJGkIIYQImCQNIYQQAZOkIYQQImCSNIQQQgQsJENuNU17HbgCOKTreqci7leYl0ccjHmJyZG6rq/3308IIUR4haqm8SbmlbyKcxnm5QtbA6OAl0MQkxBCRAfDQOWm4zi5DXavrNChQlLT0HV9haZpzUvY5SrgbV3XDeA7TdOSNU1roOv6/lDEJ4QQkUq5M7G5DmB3HcSeedD8O+sQNtdBc5vrADbXQWzuTB7++GJ+2teApX+UdD2rklWVGeGNMK/ulWePte20pKFp2ijM2gi6rpOamuq/S1RyOBxSFhYpiwJSFgUirizcLji1H2X949Q+63/fbftROSdLPxZgOOLp2BL+tbp5hcKqKkmjqDVQilxXStf1OcCcvH1kWQCTLJFQQMqigJRFgSpTFp5s7FmHC2oHroPYMs0agd11EFvWIfP/nMAummjY4vDE18MTXw+v3/+b9ifzy59xXHNdF4yYRC4eBiv+nlGh8KtK0tgDNPG53RiQRaWEEJHDm4st63BBIijUZGRtyzqIPftYQIczbDF4nHWtBFDfJxnUxWvd9sTXw4hNBlX4vNvlcjN79k+88sov2O02uvQ9ixYtFApo0iSxQi+zqiSNRcDdmqbNw7wu7wnpzxBCVAleN7asI1Y/gU/twOovyEsKtuyjqAAW3jaUHa+zjl8iMBODN76uta0+3rjaoMo+VmnZst089thq/vorHYDrr29N7drOMh+nOKEacvsuMABI1TRtD+alNWMAdF1/BfgMc7jtNswht8VebF0IISqF4cWWddT88c/ySwQu39rBYZThLf1wyoYnrk4RiaBwUvDGnQE2e6W/nP37TzFhwrd8+ukOANq3T2HGjL50716vUp8n0q/cZ8jS6KYq015bBUhZFIjKsjAMbDnHsWX6/PhnHaSGcYKc47t8RhQdRhnugA7piUu1agFF1Q6sfgRnKtjC13hz661L+OKLXcTHO3jooW7cfnsnHI6iayrW0ujReT0NIUSUMAxU7onCQ0vzO459aweHUN6cIg8R73fbE1s7v38gr2mocO2gHl5nHbDHBv/1lYPb7c1PDOPGnYvDYWPChF40alQzaM8pSUMIEV6GgXJnWKOIDhTuSC403+AgypMV0CG9MUl+NYJ6xNdpyUlPzfw+A098HbBXXlt/KJ08mcNTT61l+/YTvPPOZSilaNUqmTlzBgb9uSVpCCGCpsiJZ/lNRocKTTwLhNdR0ycZ+NYI6vrUGOphOPzrFBCXmkpWhDfVGYbBxx9vZ+LE7zh4MBO7XbFx41E6dQrd/BNJGkKIsnO7rB99KxFkFu5IzksKttz0gA7ntccXMbQ0r0ZQtyAZxASv2aWq27nzJOPHr+brr/cA0K1bXWbM6EuHDmeENA5JGkKIAqVNPMvblnsioMMVPfGs/mnbjJjE0+YaiAKvvPIrs2b9SFaWh6SkWMaNO5cbb2yHzRb6MpOkIUQ08OZic5k1g0LzDXwnnrkOYM85HtDhip94ZiWFhLxkkCTJoBK4XG6ysjwMHdqKJ57oRWrq6c1voSJJQ4hIljfxzGdoqW8zkSPnKPXS95Zt4lne0NL8pOA3tDShPt7Y5HJNPBOBOXrUxZ9/nuDcc+sDMHp0F3r3bkCvXg3CHJkkDSGqJt+JZ8UOLS3jxDNnQd+A+bf/xLN6eONSgjLxTATG6zWYN28LU6f+gN2uWL58OLVrO4mLs1eJhAGSNIQIraImnhW5lHVZJ57Vs5qE6hdKCrUatOVodlzYJ56J0v3++zHGjFnF2rUHAejXrxEul5vatcMcmB/5FAlRGYqbeJaXCLJKn3jmr/DEs3r5/QReZ+ATz4zUVLwRPsy0usvMzOW559YzZ85vuN0GderEM2lSb4YMaYmqgv1BkjSEKElJE88KzTeoyMSz+ngTfBNBvYieeCbKZtSopXz99R6Ugptv7sCjj3YnKSku3GEVS5KGiFoq95S5VHVxtYO8iWceV0DHK3biWYLPMFNnXShi4pmIXqNHd+HwYRfTp/ela9e64Q6nVJI0RPXjdmFP31kwtLSoiWeug9jcgV2MxmuPx5tQH4+ziPkGPk1GRkyNIL8wEencbi+vv76RPXvSmTz5PADOO68hn39+TVjmXJSHJA0ROTzZ2F2H8tckKmniWSCLQRt2Z+FJZs7CF7fxJph/G46aMtdAVNhPPx3i0UdXsXHjUQBuuqkdbdumAERMwgBJGqIq8J145jvfoCITz4pavjovKcjEMxFCJ05kM2PGWubO3YxhQOPGNZky5bz8hBFpJGmI4PGfeOY/36A8Vzwr7poGeXMQEuqT0rA1R44eDcELFKJkCxf+yYQJ33L4sAuHQ/H3v5/FffedQ0JCTLhDKzdJGqLsipp4dlpSqMDEsyKvaVAPr/OMwGYhS+1BVBHLl+/h8GEXPXrUY/r0vrRvH5m1C1+SNESB4iaenTbfILCJZwaq6IlnfgvXeeNSZRayqBaysz0cOHCKZs1qATB+fE969qzP8OFtIqrfoiSSNKJQ7KHviD289rShpfasQyhvbkDH8MSlWP0EfhPPfDuSnXXAFrnVcCHKYtWqvYwduxqbTfHll9cSG2snJcXJdde1DXdolUqSRpRx/vUxKav/Uez93thknx/9uqdPPIuvj8dZB+xVd/KREKF0+HAmkyd/zwcfbAOgVatk9u8vqG1UN5I0ooj95J8kf/8QAJnNh5Fbu5NMPBOinLxeg3fe+Z3p03/gxIkcnE4799xzDnfeeRaxsdW3uVWSRpRQbhcpq/6OzZ2Bq+mVpPWaLR3GQlTAbbd9yZIluwAYMKAxU6f2oXnz6lm78CVJI0rUWjeemBObcSe2JO3cpyVhCFFBl13WnJ9/PsSkSb258sqqubhgMEjSiALx2+dTY/s8DLuTY31ejerrLAtRXkuW7GLfvlOMHNkBgOHDWzN4cHNq1ix+leHqSJJGNedI20zSj+MASOs+DXftDmGOSIjIsndvBo8/voYvvthFXJydCy5oTLNmtVBKRV3CAEka1ZrKTSdl1ShsniwyW1yHq+V14Q5JiIiRm+vltdc28Mwz68jMdFOzZgyPPNKdxo2ju6YuSaO6MgySf3gYR/p2cpPac6L71HBHJETEWLfuII8+uorNm48BcMUVLZg4sTcNGshKxpI0qqmErW8R/9fHeB01ONb3FQwZSitEwGbNWsfmzcdo2jSRKVPO46KLmoY7pCpDkkY1FHP0Z5J+mghAWs+n8dRqFd6AhKjiDMMgIyOXxESzj2LKlPN4772t3HvvOcTHy8+krwBWfxORRGUfp/bqv6O8uWS0voWspkPCHZIQVdq2bWlcd91n3H77lxiGudpyq1bJjBnTQxJGEaREqhPDS+3v7sNxag85KWdz8pzHwx2REFVWVpabF1/8hX//+2dycrzUrh3H7t3pNG1a/SfoVYQkjQjgOLGVuH1fQQnXnLDVSCBp/6849y3FG5vM8T6vyPpQQhRjxYo9jB27mp07TwJw/fVteOyxnqSkOMMcWdUXsqShadog4HnADvxX1/UZfvc3Bd4Ckq19xui6/lmo4qvKkr+/n9ijP5W6X96bebzXbDw1mwQ3KCEikGEYPPjgCubP/wOANm2SmTGjLz17NghzZJEjJElD0zQ78G/gYmAPsFbTtEW6rm/y2W08oOu6/rKmaR2Az4DmoYivqlM55tlQZovr8MbVLnKf+Ph4XC4X2fXOI7vhRaEMT4iIoZSiSZNEnE4799/flVGjOlfrxQWDIVQ1jXOBbbqubwfQNG0ecBXgmzQMIK8xMQnYF6LYIkZGh9G4ixkJFZuayskjR0IckRBV34YNRzl0KBNNSwVg9OguDB3aSvouyilUSaMRsNvn9h6gp98+E4Elmqb9E6gBDCzqQJqmjQJGAei6TmpqaqUHW9U47OaZUHJyMqQU/XodDkdUlEUgpCwKRHNZpKdnM3nySl588UfOOCOeQYM65ZdFo0ZhDi6ChSppFLX8o3+v7g3Am7quP6NpWm9grqZpnXRdL3SRaV3X5wBz8o5xJArOrut4PMQAaWlpuL1Fv97U1FSioSwCIWVRIBrLwjAMFi/eyeOPf8v+/aew2RRDhrREKW/UlUVxGjZsWO7Hhmqexh7At2e2Mac3P90G6AC6rn8LOIHoPEUSQpTLnj3pjBy5hNtvX8r+/afo0iWVzz67msmTe5OYKKMJK0OoahprgdaaprUA9gLXAzf67fMXcBHwpqZp7TGTxuEQxSeEiHCGYXDHHUv59dcjJCbGMGZMD0aMaI/dLnOYK1OZS1PTtLplfYyu627gbuALYLO5Sd+oadpkTdPypiw/CNyhadovwLvASF3Xi5+YIIQQmJddBXNk1OOP92TIkJYsX64xcmRHSRhBoPKmzZdE07Qk4AVAAzy6rtfQNO1KoLuu6xOCHGNJjH37qv8gqzqf9CMm/U8OXb682NFT0dh2XRwpiwLVuSyOHcti+vQfAJg1q1+p+1fnsigrq0+jXJcaDDQNvwxkA62BHGvb95id10IIETKGYaDrf9C//wL+978tvPfeVvbtywh3WFEj0KQxELhL1/XdWKOedF0/BNQLVmBCCOFv69bjDB/+Kfffv5xjx7Lo3bsBX345lIYNo/vCSKEUaEf4SSAFOJC3QdO0JsDBYAQlhBC+DMNg1qx1vPTSL+TmeklJcfLEEz0ZNqw1SpWrlUWUU6A1jdeBBZqmnQ/YNE3rAbwBvBq0yIQQwqKU4sCBU+TmernppnasWDGc4cPbSMIIg0BrGtMx+zJewxwK+z/MhPFckOISPpTHBYChZI0cET0OHDjFsWNZdOhwBgDjx/fkhhva0qNH/TBHFt0CTRpn6Lr+NPC070bNXMxFhiMEkco5gSNzH4YtDk+NxuEOR4ig83i8vP32ZmbOXEv9+jVYsuRaYmPtpKQ4SUmRhBFugSaN7RQsJujrD8y+DhEkMcc3ApCb3A5sMWGORojg+u23Izz66Ep++cU8F+3ZsxYZGbmkpEgtu6oINGmc1nCoaVpNwFvEvqIS5SeN2p3CHIkQwZOensOsWT/yxhub8HoNGjSowZNP9mbQoObSb1HFlJg0NE3bgTnENl7TtO1+d6cC7wcrMGGKOb4BgNzaHcMciRDBYRgG1177MZs2HcNuV4wa1ZkHH+xKzZqx4Q5NFKG0msbtmLWMRcAdPtsN4KCu6xuDFZgwFdQ0JGmI6kkpxR13dObttzcxY8b5dOp0RrhDEiUoMWnouv4VgKZp9XVdPxmakEQ+TxaOk1sxULiTO4Q7GiEqRU6OhzlzfsNuV9x5ZxcAhg9vzdChrWStqAgQUJ+GrusnNU3rBJyP2SylfO6bHKTYol7MiT9QhpvcWq0wHAnhDkeICvv++/2MGbOKP/5IIy7OzrBhralTJwGlFHa79F1EgoCShqZpt2EuWPgV5nW+v8Rcxvzj4IUm8vszkqVpSkS2Y8eymDLle+bP/wOAFi1qMW1aX+rUkZOhSBNoXXAMMFjX9SsBl/W/BpwKWmQivz/DLSOnRIQyDIP587fQr5/O/Pl/EBtr44EHurJ06VD69ZNrrkaiQIfc1tN1/Rvrb6+maTbgU+Bt4NZgBCZ8R05J0hCR6/33t3H8eDZ9+jRk2rQ+tGqVHO6QRAUEmjT2aJrWTNf1XcBW4HLMmeC5QYss2nk9ONI2ATJySkQWl8vNyZM51Ktn9lVMm9aHX345zLXXtpI5F9VAoEnjGaATsAuYAiwAYoAHghRX1LNn7MTmzsQTXx+vU4YgisiwbNluHntsNU2bJjJv3mCUUrRqlSy1i2ok0NFTr/n8/YmmabWBOF3XTwQtsignTVMikuzff4oJE77l0093AFCjRgzHj2eTkuIMc2SispVrULSu61mAQ9O06ZUcj7DIpD4RCTweL6+9toEBAxbw6ac7SEhw8MQTPVm8+BpJGNVUqTUNTdNuBs7G7MuYAyQAjwP/ANYENbooJmtOiarO6zUYOvQT1q41r8U2aFAzJk8+j0aN5Cp61Vlpa089BYzATA43AL2A3sA6oK+u678EPcJoZBjSPCWqPJtN0b9/Y/buzWDq1D5cckmzcIckQqC0msb1QD9d17dqmtYe2AjcoOv6/OCHFr1sroPYs4/gjamFp0aTcIcjBGDOuVi0aDsOh43LL28BwOjRXRg1qjM1asiy/dGitD6NZF3XtwLour4ZyJSEEXwxaT79GTJEUVQBO3ee5KabPmf06GWMHbuKtLRsAOLi7JIwokxpNQ2laVoTCtaacvvdRtf1v4IVXLSS5dBFVZGd7eHll3/hhRd+JivLQ3JyHI8+2oNatWTZ8mhVWtKoAeyk8EWYdvn8bQBySa1KJv0ZoipYs2YfY8euZtu2NACGDm3FE0/0IjU1PsyRiXAqLWlIvTMMYo5bM8FloUIRJh6Pl3HjzIRx5plJTJ/elz59GoY7LFEFlHY9DU+oAhEmlXMSR8ZODFsc7qTW4Q5HRBGv1yA720N8vAO73cb06X357rv9jB7dhbg4aVAQpkCXEREhEpO33lRyW7BJRU+ExubNxxgzZhWtWiXxzDP9AejduwG9ezcIc2SiqpGkUcXkT+qTpikRApmZuTz33HrmzPkNt9tg9+500tKySU6OC3doooqSpFHFSCe4CJUlS3Yxfvwa9u7NQCm4+eYOPPpod5KSJGGI4gWcNDRNcwA9gEa6rr+naVo8gK7rrmAFF41kuK0INrfby513fsVnn+0EoGPHM5g5sy/nnFM3vIGJiBDo5V47Agutm/WB9zAv93oT5vIiojJ4cnCc3IqBwp3cIdzRiGrK4bCRmBhLjRoxPPxwN265pSMOR7nWLhVRKNCaxsvAFF3X39Q07bi17RvglUCfSNO0QcDzmPM6/qvr+owi9tGAiZjzP37Rdf3GQI9fHThO/OnnkQkAACAASURBVIHy5uJObIkRUyPc4YhqZP36QwB07WrWJsaP78lDD3WjYUNZXFCUTaCnF52Bt6y/DQBd1zMwV7wtlaZpduDfwGVAB+AGTdM6+O3TGhgL9NF1vSNwX4CxVRvSnyEqW1paFmPGrGLIkIU88MBycnLMUfQpKU5JGKJcAk0au4BzfDdomtYd+DPAx58LbNN1fbuu6znAPOAqv33uAP6t6/pxAF3XDwV47GqjYM0pSRqiYgzD4MMPt3HWWXOYO3czdrvikkua4fEY4Q5NRLhAm6eeAD7VNO0lIFbTtIeBu4A7A3x8I2C3z+09QE+/fdoAaJq2GrMJa6Ku64sDPH61IJ3gojJs336CceNWs3LlXgB69KjHjBl9adcuJcyRieog0Mu9LtI0bT9mbWA10BbQdF3/IcDnKWqpVv9THgfQGhgANAZWaprWSdf1NN+dNE0bBYyy4iI1NTXAEKo4w5s/sS/xzH4kJpTtdTkcjupTFhUUzWWRm+vhhhvmsWdPOikpTmbMGMiIEZ2w2WS15Gj+XFSmQEdP1dZ1fS2wtpzPswfwvTBEY2BfEft8p+t6LrBD07QtmEmk0HPquj4H8wqCAMaRI0fKGVLVYk/fQb3cDDzx9TmSaYPMsr2u1NRUqktZVFQ0loVhGChrGf2HHurKmjX7GT/+XNq2bRJ1ZVGcaPxcFKdhw/KvIxZo89ReTdOWAu8Ai8oxN2Mt0FrTtBbAXsyLO/mPjPoIc/jum5qmpWI2V20v4/NErIKmKRlqKwJ3+HAmkyd/T8uWSdx/f1cAhg9vw/DhbcIcmaiuAu0IbwEsBe4HDmqaNlfTtMusUVGl0nXdDdwNfAFsNjfpGzVNm6xp2hBrty+Ao5qmbQK+Bh7Wdf1oWV5MJJORU6IsvF6DuXM307//Aj74YBv/+c9vZGTkhDssEQWUYZRtNIWmaS0xawk3AKm6rtcLRmABMvbt82/likwp34zAuX8Zx/rOIavJ5WV+vFS9C1T3sti48ShjxqzKn3txwQWNmTq1D82a1Tpt3+peFmUhZVHAap4qV0dXedaeSrL+JQKnyvOk4nT5NQ1ZqFAUIzfXy/TpP/Df/27A4zGoVy+BSZN6c8UVLfL7M4QItkA7wttg1ixuxEwYC4DrdV1fE8TYoobNdQh71iG8MYl4ajYNdziiinI4FBs2HMXrNbj11o48/HB3ueyqCLlAaxprgQ+Be4ClcnGmylWolqFkDSBRYO/eDDweL02b1kIpxYwZfUlPz6FLlzrhDk1EqUCTRj1d17OCGkkUy7+GhkzqE5bcXC+vvbaBp59eR7dudZk3bzBKKVq2TAp3aCLKFZs0NE27Qdf1dwtuakXup+v628EILJrITHDh68cfDzJmzCo2bz4GQHJyHC6Xm4QEuZKjCL+SahojgbykcUcx+xiAJI0KkuG2AiAtLZtp037gnXd+B6Bp00SmTu3DhRc2KeWRQoROsUlD1/VLff4+PzThRB+Vm4EjYyeGLRZ3rdbhDkeESXa2h0su+YC9ezOIibHxj3+cxb33nkN8vFxcU1QtAfW6appW5PIhmqZ9V7nhRJ+89aZyk9qAXUbCRKu4ODs33NCWXr3qs2TJtYwZ00MShqiSAh2q066Y7bJWQQVJ01R0yspy8/TT6/jww2352/75z7N5770raNOmdhgjE6JkJZ7KaJr2uvVnrM/feZpjLgkiKsBxXK6hEW1WrNjD2LGr2bnzJKmp8Qwa1Jz4eIdcclVEhNLqv3uL+dsA1gHzKz2iKJNX03DLyKlq79ChTCZN+o6PPjKvXda2bW1mzOgrzVAiopT4adV1/XEw+y50Xf80NCFFEU8OMSe2YKDITZbVbasrj8fL3Lm/M3PmWk6ezMHptPPAA125447OxMYGtOanEFVGSfM0+ui6vtq6ma5pWr+i9tN1fUVQIosCjpNbUd5c3IktMGLkes3Vlcdj8MYbGzl5MocLL2zC1Knn0bTp6YsLChEJSqppvEZBB/g7xexjALJYUjnJIoXVV0ZGDh6PQVJSHLGxdmbNOp/Dh10MHtxcFhcUEa2keRrtfP6W2UVl4Di+gVq/zER5Sl55xX7KvGy6dIJXH4Zh8PnnO3n88W8ZMKARzzzTH4Bzz60f5siEqBzl6oHTNO18wK3r+reVHE+1UOvXp3DuXxbw/tn1egcxGhEqu3enM378GpYu/QuALVuOk5XlxumUjm5RfQS6NPo3wOO6rq/UNO0h4FHArWnabF3XZwYzwEhjyzxA3P6vMZSDY/1ex7DFlbi/15mKO7m4aTAiEuTmepkz51eefXY9WVkeEhNjGDOmByNGtMdul2G0onoJ9BSoM5BXq/g7MABIB1YCkjR8JOx8H2V4cTW+lOyGF4U7HBFkLpebK69cmL+44FVXncmECb2oVy8hzJEJERyBJg0b4LUu9erQdX0jgKZpKUGLLBIZBvHbzakrmS2vC3MwIhTi4x2cdVYqLpebadP60L9/43CHJERQBZo01gCzgYaYF2PKu1b40SDFFZFijvxITPqfeJx1yW5wQbjDEUFgGAYLFmylefNa+Z3bEyf2JibGJpP0RFQItMF1JJAFbAEmWNs6AC8EIaaIlbBDB8DVfCjY5Aekutm69TjDh3/K/fcv55FHVpKTY17AslatWEkYImoE9EnXdf0w8Ijftk+AT4IRVCRS7kzidy0CpGmqunG53PzrXz/x8su/kpvr5YwznNx999nExEgnt4g+gY6ecgBjgRFAI8x1qOYCM3Rdzw1eeJHDuftTbO4Mcs7oijtJrotRXXz99W4ee2w1u3alA3DTTe0YO7YHtWs7wxyZEOERaJ16JtAHuA/YBTQDxgPJwIPBCS2yJEgHeLVz6lQu99zzDceOZdGunbm4YI8eMklPRLdAk4YGnKPr+hHr9kbrwkw/I0kDe8Yu4g59i9fuxNV0SLjDERXg8XjxeiEmxkaNGjFMntyb/ftPcccdnaU5SggCTxp2wOu3zQvIIjpAwnazAzyryWCMWFmILlL9+uthHn10FZdc0oz77+8KwDXXtApzVEJULYEmjfeARZqmTQD+wmyeegJ4P1iBRQyvh3hr1FRmy+vDHIwoj/T0HGbN+pE33tiE12uQnp4jHd1CFCPQpPEw5lDb14AGwD5gHjApSHFFjLiDq3Fk7sNdowk5dWUNqUhiGAaffLKDCRO+5eDBTOx2xahRnXnooW6SMIQoRqBDbrOBcdY/4SN+h9UB3kIDJT80kSIjI4c771zGsmXmSsPnnFOXGTP60qnTGWGOTIiqrbRrhLfGrF10AtYDt+q6/lcoAosEKieN+N2fY6BwtdTCHY4ogxo1YsjO9lCrVixjx/bgb39rj80mXXRClKa0msaLmHMyngZuxFxK5NpgBxUp4nctRHmzya7XF08NWXOoqvvuu/3UrZtAy5ZJKKV49tl+xMXZqVNHFhcUIlCltad0w6xdLALuAHoGP6TIkTdqSuZmVG3HjmXxwAPLGTr0E8aOXYVhGAA0bpwoCUOIMiotacTquu4C0HU9HYgPfkiRwZH2O7HHfsYbUwtX48vCHY4ogtdrMG/eFs4/X2f+/D+IjbXRs2d9PB4j3KEJEbFKa56K0zTtCZ/b8X630XV9ciBPpGnaIOB5zDkf/9V1fUYx+w0DFgA9dF3/MZBjh0PeDHBXsyHgkFxa1WzZcoyxY1fz/fcHAOjbtyHTpvXhzDOTwxyZEJGttKShA74LKb3ndzugUzZN0+zAv4GLgT3AWk3TFum6vslvv0TgHuD7QI4bNt5c4neaU1RkbkbVc+JEFldeuYhTp3JJTY1nwoReXHPNmSglHd1CVFSJSUPX9RGV9DznAtt0Xd8OoGnaPOAqYJPffk8CTwEPVdLzBoVz31fYs4+SW6sNuSlnhzscYTEMA6UUSUlORo8+iwMHMhkzpgfJySVfclcIEbhQXQSgEbDb5/Ye/DrVNU07B2ii6/on1nXIi6Rp2ihgFICu66SmpgYh3JI5vvsQANX5FlLr1An58xfF4XCEpSyqgr1703nwwS+58srW3HRTZxwOB08+ebHULIjuz4U/KYvKEaqkUdS3N79pS9M0G/Ac5sWeSqTr+hxgTt4xjhw5UtLulc7mOkS9nZ9jKDtH6lyGN8TPX5zU1FRCXRbh5nZ7efPNTTz11I+cOpXLunX7GDiwHvXq1eXoUbmoJETn56I4UhYFGjZsWO7HhmoK8x6gic/txphLkeRJxJxA+I2maTuBXphrXXUPUXwBi9/5AcrwkNXwIrzxVaOWEY1+/vkwV1yxkAkTvuXUqVwGDWrGe+9dgd0us/KFCKZQ1TTWAq01TWuBOVnweszJggDoun4CyK83apr2DfBQlRs9ZRgFo6akAzwsMjNzmTr1B956axOGAY0a1WTKlPO45JJm4Q5NiKgQcNLQNO0CzB/7erquX61pWlcgUdf15aU9Vtd1t6ZpdwNfYA65fV3X9Y2apk0GfrQmD1Z5MUd/IubkH3jiUslqeGG4w4lKdruNlSv3YrOZiws+8EBXEhJiwh2WEFEj0Mu9jsYc0fQ6ZuIAyAGmAn0DOYau658Bn/lte6KYfQcEcsxQS7AWJ3Q1vxZs8kMVKjt3nqRWrVhSUpzExdn5178uIC7OTvv2KeEOTYioE2gD8IPAQF3Xp1BwMabNQPugRFUFKbeL+F0LAVk2JFSysz3Mnr2eiy56j2nTfsjffvbZdSRhCBEmgTZPJWJeGxwKRj05MGsbUcG553NsuenkpJyNO7lduMOp9tas2cfYsavZti0NMEdKeTxe6egWIswCTRqrMJunZvpsuwsotT+jusjrAM+UJdCD6sgRF08++T3vvbcVgDPPTGL69L706VP+IYJCiMoTaNL4J/CJpml3AImapm3ErGUMDlpkVYg9YzexB1dj2J24ml0d7nCqrWPHsujffwFpadnExdn55z/PZvToLsTF2cMdmhDCEuiV+/ZqmtYN6A00xZzd/a2u655gBldVxO9YgMIgs/EgjNikcIdTbaWkOLn00mbs33+KadP60KKFlLUQVU3AQ251XfcCq61/0cPwkrBDrpsRDJmZuTz33HouuqgpvXo1AGDatD7ExdllCRAhqqhAh9zuoJgVbXVdb1mpEVUxsQfX4Di1G3dCI3LqBTS6WARgyZJdjB+/hr17M/jqq90sXToUm03hdIZqvqkQojwC/Ybe7ne7AWY/x7uVG07Vkz83o8VwUDJyp6L27s1gwoRv+fzznQB06nQGM2eeL9fnFiJCBNqn8ZX/Nk3TvsKcrDe7soOqKlTOSZy7zfmIMmqqYtxuL6+9toGnn15HZqabGjVieOSR7owc2QGHQ5KxEJGiIm0BLqBaN03F/7UImyeL7Lq98dSUtY0qIj09hxdf/IXMTDeDB7dg0qReNGxYM9xhCSHKKNA+Df/lPhKAy4EllR5RFVIwN0M6wMvjxIlsnE4HcXF2atd2MnNmX2Jj7Qwc2DTcoQkhyinQdoHWfv+SMS/fWllX9qtyHCe2Ent0PV5HTbKaXB7ucCKKYRh8+OE2+vVbwEsv/ZK/ffDgFpIwhIhwpdY0rOt7fwnouq5nBT+kqiF/CfRmQzAcCWGOJnL8+Wca48atZtUq83Ip339/IP8yrEKIyFdqTcOawPdCNCUMvLnE73wPkKapQGVluXn22XUMHPg+q1btIzk5jmee6cf//neZJAwhqpFAm6c+1TQtKpYMAYjb/zX2rMPkJp5J7hndwh1OlXfoUCYDB77PM8+sJyfHi6a1YeVKjeuvbytDaYWoZgIdPWUDPtA0bRXmEiL5E/10Xb81GIGFU8J2cwa4q+V1IGfJpapTJ56GDWvicNiYPr0vvXs3CHdIQoggCTRpbAVmBTOQqsKWdRTn3i8xlJ3MFsPCHU6V5PUavPPO75x3XgPOPDMZpRT//vcFJCXFERsriwsKUZ2VmDQ0TbtB1/V3dV1/PFQBhVv8zvdRhpushhfhja8X7nCqnI0bjzJmzCrWrz9E374NmTdvMEop6tSRwQJCRIPSahqvEgVLheQzjPymKekAL+zUqVyeeWYd//3vBjweg/r1ExgxImou3CiEsJSWNKKqQT/m2K/EnNiMJy6FrIYXhzucKmPx4p2MH7+G/ftPYbMpbr21I4880p3ExNhwhyaECLHSkoZd07QLKCF56Lq+rHJDCp/8xQmbXQN2+UEE2L//FKNHLyM728NZZ6UyY0ZfunSpE+6whBBhUlrSiANeo/ikYVBd1p/yZBG/8yNAmqZyc704HAqlFA0a1OCRR7oTG2vj5ps7yDW6hYhypSWNU9X9ehl5nHu+wJZ7gpzanXHX7hjucMJm7dqDjB27in/84yyGDWsNwD/+cVaYoxJCVBVy2miJ9sUJjx/P4pFHVnL11YvYvPkYb721CcMo8rpbQogoJh3hgP3UXuIOrMCwxeJqdnW4wwkpwzB4//1tTJ78HUePZhETY+POO8/innvOkeU/hBCnKTFp6LqeGKpAwil+xwIUBq7Gl2LE1Q53OCFz+HAmo0cvY82a/QD07t2A6dP70Lp19JSBEKJs5ILMhpeEHXlzM64PczChVatWHIcOuUhJcfL44z0ZPry11C6EECWK+qQRe/h7HBm78CQ0ILve+eEOJ+hWrNhDp06ppKQ4iYuz8+qrF1G3bgIpKc5whyaEiABR3xGe3wHefBjYqu+6SQcPmk1RN9zwOdOm/ZC/vV27FEkYQoiARXVNQ+Vm4PzrE6D6jpryeLzMnfs7M2b8QHp6Lk6nnTPPTJILIwkhyiWqk0b8Xx9j87jIrtMTT2KLcIdT6X777Qhjxqzi558PA3DRRU2YOrUPTZpExfgGIUQQRHfSqMZzM3bvTufyyz+yFheswZNP9uayy5pL7UIIUSEhSxqapg0CngfswH91XZ/hd/8DwO2AGzgM3Krr+q5gxWM/uY24I2vxOhLIanJFsJ4mbJo0SeS669pQo0YMDz3UjZo1ZS0tIUTFhaQjXNM0O/Bv4DKgA3CDpmkd/Hb7Ceiu6/pZwHvAU8GMKWH7AgCymlyJEVMjmE8VEjt3pnHzzV/w7bf787c99dT5TJzYWxKGEKLShKqmcS6wTdf17QCaps0DrgI25e2g6/rXPvt/B/wtaNF43STsfA+AzDMje25Gbq6XOXN+5bnnfsLlcnPsWBYff3wVgDRFCSEqXaiSRiPMa4vn2QP0LGH/24DPi7pD07RRwCgAXddJTU0tczBq5+fYXQcwkltRq91lEXsd8NWrd3P33YvZtOkIAJrWnqeeGkhqas0wRxZeDoejXJ+L6kjKooCUReUIVdIo6le5yNXwNE37G9Ad6F/U/bquzwHm5B3jyJEjZQ6m9s//IQZIbzqUjKNHy/z4cEtLy2bKlO95990tADRvXosXX7yMc86pBWRx5EhWeAMMs9TUVMrzuaiOpCwKSFkUaNiwYbkfG6qksQdo4nO7MbDPfydN0wYCjwH9dV3PDkYgtuxjOPcuwVA2MlsMC8ZTBJ3Xa/DFF7uIibFx111duPvus2nSpL58IYQQQReqpLEWaK1pWgtgL3A9cKPvDpqmnYN5TfJBuq4fClYg8Ts/RHlzyWpwAd6E8mfbUNu2LY0mTRKJi7OTkuLkxRcvoFGjmrRqlRzu0IQQUSQko6d0XXcDdwNfAJvNTfpGTdMma5o2xNptFlATWKBp2s+api0KRiwF183QgnH4SudyuZkxYy0DB77PSy/9kr+9f//GkjCEECEXsnkauq5/Bnzmt+0Jn78HBjsGx7ENxKRtxBubTFajS4P9dBX29de7GTduNX/9lQ6YF0oSQohwiqoZ4Qk7rFpGs2vAHhfmaIp34MApJkz4lk8+2QFA+/YpTJ/elx496oU5MiFEtIuepOHJJmHnB0DVvm7Gn3+mMXjwR2Rk5BIf7+DBB7ty++2diYmJ+gWJhRBVQNQkDefeJdhy0shN7oA7pVO4wylWy5ZJdOlSh4QEB1OmnEfjxrK4oBCi6oiapJGwPe/qfFVrccL09BxmzVrHzTe358wzk1FK8eabl5CQEBPu0IQQ4jRRkTRsmfuJO/ANhi0GV/Nrwx0OAIZh8MknO5gw4VsOHszkzz/TeOedywAkYQghqqyoSBoJO95DGV5cjQfhjUsJdzjs2nWS8ePXsGyZubJK1651GTfu3DBHJYQQpav+ScMwCkZNtQhv01ROjodXXvmV55//iawsD0lJsYwdey433dQOmy0y178SQkSXap80Yo+sxZG+A098PbIbDAhrLPv2nWL27J/IzvZw7bWteOKJntSpkxDWmIQQoiyqfdLIvzpf82FgC/3LTUvLJikpFqUUzZvXYtKk3jRvXovzz28U8liEEKKiqvXgf5V7ivi/PgZCv2yI12swb94W+vSZz/vvb8vfPmJEe0kYQoiIVa2ThnP3J9jcp8hJ7Y6nVquQPe+WLccYNuwTHnxwBWlp2Xz99e7SHySEEBGgWjdPhXpuhsvlZvbs9bzyyq+43QapqfFMnNiLq68+MyTPL4QQwVZtk4Y9fQdxh7/Da4/H1fTKoD/fn3+mcdNNn7N7dwZKmc1QY8b0IDm56q5xJYQQZVVtk0bCDrOWkdXkcoyY4C/F0bhxInFxDjp0SGHGjL506yaLC4oChmGQlZWF1+sN6bXbDx48SHZ2UK5nFnGirSwMw8Bms+F0Oiv1M1c9k4bXQ8KOBQBknhmcxQndbi9z527mqqvOJCXFSVycnXfeGUT9+jVwOKp1V5Eoh6ysLGJiYnA4QvuVczgc2O32kD5nVRWNZeF2u8nKyiI+Pr7Sjlktk0bcwZXYM/fjrtmMnDq9Kv34P/10iDFjVrFhw1E2bjzK00/3A5DFBUWxvF5vyBOGEA6Ho9JrV9XyU5x/db4Ww6ESq2UnT+Ywc+Za3nprE4YBjRrV5JJLmlXa8UX1FcomKSF8VfZnr9olDZV9HOeexRgoXC0qZ26GYRgsWrSdiRO/5dAhFw6HYtSoztx/f1dZXFAIEVWqXeN7/K6FKG8O2fXPx1OjcibRbdx4jNGjl3HokIvu3euxePG1PPZYT0kYIqI0adKEiy++mAsvvJCbb76ZEydO5N+3ZcsWhg8fTt++fenTpw/PPfcchmHk379s2TIuu+wy+vfvT79+/Zg8eXI4XkKJNmzYwEMPPRTuMEr0wgsv0KdPH84//3y++eabIvdZuXIll156KRdffDFXX301O3aYV/B8++23ueiii/K3//HHHwBs3ryZ++67L1QvofoljbzFCV0VXJzQ4/Hm/92p0xnccUcnZs06nw8/vJL27cO/Uq4QZeV0Ovnyyy9ZtmwZycnJvPnmmwC4XC5uueUW7r77blatWsXSpUtZt24db731FgC///4748eP54UXXmD58uUsW7aMpk2bVmpsbre7wsf417/+xS233BLS5yyLP/74g4ULF7Js2TLeeecdxo0bh8fjOW2/sWPH8uKLL/Lll19y9dVX8/zzzwNwzTXX8NVXX/Hll18yevRoJk2aBED79u3Zv38/e/fuDcnrqFbNU47jm4g99ivemCRcjS8t93FWr97HuHGrmTmzL716NQBg4sTelRWmiHIN3w3OMjL7bgj8R6Nbt25s3rwZgI8++oju3bvTv39/AOLj45kyZQrDhg1j5MiRvPTSS9xzzz20amWuquBwOBg5cuRpxzx16hTjx4/n119/RSnF/fffz+WXX07r1q3ZunUrAJ988glLly5l9uzZ3HfffSQnJ7NhwwY6duzI4sWLWbJkCUlJSQD06dOHjz76CJvNxpgxY/J/FCdNmkSPHj0KPXdGRgabN2+mY8eOAPz0009MmDCBrKwsnE4nzz77LO3atWP+/Pl89dVXZGdnk5mZyYIFC3j55Zf5+OOPycnJYdCgQfm1lVtvvZV9+/aRnZ3Nbbfdxt/+9reAy7coX3zxBVdddRVxcXE0bdqU5s2b89NPP9G9e/dC+ymlSE9PByA9PZ169czh+4mJBQNtMjMzC/VVXHzxxSxcuJDRo0dXKMZAVKukkV/LaHYVOMo+xOzIERdPPvk9771nfsDnzPktP2kIUV14PB5WrVrFDTfcAJhNU2eddVahfZo3b05mZibp6els2bKFv//976Ued/bs2SQmJvLVV18BkJaWVupjtm/fzvz587Hb7RiGweLFi7nuuutYv349jRs3pk6dOtx1113ccccdnHvuuezdu5cbb7yR5cuXFzrOL7/8Qrt27fJvt2rVig8++ACHw8GKFSuYOXMmb7zxBgDr1q1j6dKl1K5dm+XLl7Njxw4+/fRTDMNg5MiRfPfdd/Tq1YtnnnmG2rVr43K5uPzyyxk8eDApKYVbGSZMmMCaNWtOe11XXXUVd999d6FtBw4coGvXrvm3GzRowIEDB0577NNPP82IESNwOp0kJiby8ccf59/35ptvMmfOHHJyctB1PX97ly5dePHFFyVplIknh/idHwCQ2bJsczO8XoN3393CtGk/kJaWTVycnXvuOZs77+wSjEhFlCtLjaAyZWVlcfHFF7Nnzx46d+5Mv37mUHHDMIodYVOWkTcrV67kpZdeyr+dnJxc6mOuuOKK/LkTV155JbNnz+a6665j4cKFDBkyJP+4ee33YNYqMjIyqFmzZv62Q4cOFfpBP3nyJPfddx87duxAKUVubm7+ff369aN27doALF++nOXLl3PJJZcA5hn8jh076NWrF6+//jqff/45APv27WPHjh2nJY28JqJA+PYR5SmqfP/zn/8wd+5cunbtyssvv8ykSZN4+umnARg5ciQjR47kww8/5Pnnn89vujrjjDM4ePBgwLFURLVJGs59S7FnHyM3qR25KWeV/gDLX3+d5J///IYffzQLvH//Rkyd2ocWLZKCFaoQYZHXp3Hy5Eluvvlm3nzzTW677Tbatm3Ld999V2jfXbt2kZCQQM2aNWnTpg2//fZbftNPcYpLPr7b/OcMJCQUXE+me/fu7Ny5BvXregAADopJREFUk6NHj/LFF19w7733AuYcl0WLFpU4Qc3pdBY69qxZszjvvPN47bXX2L17N8OGDSvyOQ3D4O6772bEiBGFjrdmzRpWrlzJxx9/THx8PMOGDStyvkNZahoNGjRg3759+bf379+f3/SU5+jRo2zatCm/RjJkyBBuuummIo8/duzY/NvZ2dk4nc7T9guGatMRnj83o6VWprkZNWvGsn37CerWjeelly7knXcuk4QhqrVatWrx5JNP8sorr5Cbm8s111zD2rVrWbFiBWB2jD/++OP5TR133nknL7zwAn/++Sdg/oi/+uqrpx23f//++U1AUNA8VadOHbZu3YrX62Xx4sXFxqWUYtCgQUycOJHWrVvnn9X3798/v9MezFFS/lq3bs3OnTvzb6enp1O/fn2AQs04/gYMGMD8+fM5deoUYP6QHzlyhPT0dJKSkoiPj2fbtm2sX7++yMdPmjSJL7/88rR//gkD4JJLLmHhwoVkZ2fz119/sWPHDs4555xC+yQlJXHy5Mn8sl6xYgWtW7cGzKa8PEuXLqVFixb5t7dv307btm2LfZ2VqVrUNGyug8TtX4ahHLiaDy11/2++2U3v3g2Ji7OTkuLkjTcuoU2b2tSqFRuCaIUIv06dOtGhQwcWLlzIsGHDeP3113n88cd57LHH8Hq9DB06NH8kUocOHZg4cSJ33XUXLpcLpRQXXXTRace89957GTduHBdeeCE2m40HHniAwYMHM3bsWG6++WYaNmxI27Zt83+gizJkyBAGDx7Mc889l7/tySefZNy4cQwcOBC3203Pnj2ZOXNmoce1atWK9PT0/GarO++8k/vuu485c+bQp0+fYp+vf//+bN26Nb8pLCEhgRdeeIEBAwYwd+5cBg4cSMuWLQv1RZRX27ZtufLKK7nggguw2+1MnTo1v2luxIgRzJo1i/r16zNr1ixGjRqFUork5GSeeeYZwOzPWLlyJQ6Hg6SkJGbPnp1/7DVr1hT5ngSDKqqdLYIY+/bto+aml6j1y1RcjQdx/PzXit15794MnnhiDYsX7+Lhh7tx330V/yBUFampqRw5ciTcYVQJVbEsMjMzCzWLhIrD4Qj50NJwmTNnDjVr1uTGG28s8v7qWhbZ2dkMHTqUjz76qMilaor67DVs2BCgXFPFI795yjCI35G3bEjRczPcbi+vvvorAwYsYPHiXdSoEUNycmja/4QQofF///d/xMZGX2vB3r17GTduXMjWNov45qmYo+uIObkNj7MO2Q0vOO3+desOMmbMKjZtOgbA4MEtmDy5Nw0a1Ah1qEKIIHI6nYU6vKNFy5YtadmyZcieL+KTRt7V+VzNh4Kt8LIe69cf4qqrFmEY0KRJTaZM6cPAgZU7k1WIQER4M7CIYJX92Yv4pBG/ayFQ9CVdzzmnDgMGNKZjx1Tuu+8c4uMj/uWKCGWz/f/27jzGqvKM4/h30DDUDSLU2ojY1gJxKSkBrKnVLmij1i2N/SFqqhVwaWzjgi2NGy51bWis4i4qLujPmipSWwkgNS5YtRWLWiwCKmrVIlJRAQftH++LczvOzD0gc2bu5fkkN5l77nvPeeaZO+e95z3nPG83mpqaojx6KFVTUxPdum3YsxA1/wnu1rSC1b0H09RzAAsXLmf8+Mc4++zd2XHHXjQ0NDB58r506xZlqUPn6tGjBytXrmTVqlWllklvbGzcqGara8/GlovKmfs2pJrvNACW9RUTJjzFFVfMZdWqNTQ2bsp11+0NEB1G6BIaGho26OxpRXXFK8k6S+Riwyit05C0L3AZsAlwve2LWrzeCEwGhgBLgRG2F1db74wXB3LcxO4sXJRuvhkxYgBnnPGNDRx9CCEEKOmSW0mbABOB/YCdgZGSdm7RbBSwzPZXgd8CF1PAPleNZOGid+nfvxd3330AEyZ8m623jstpQwihI5R1n8ZuwALbC22vBu4ADm7R5mDg5vzz74HhkqqOLfVobGDcuGFMn/7DqEgbQggdrKzhqe2AVyqeLwFajiF90sZ2k6TlQG/g/wYhJR0LHJvb8cHKszoq5pqT7/IMRC4qRS6aRS4+u7KONFo7Ymh58XCRNti+1vZQ20MlPZXft9E/IheRi8hF5GIdc7Feyuo0lgDbVzzvC7zWVhtJmwI9gbdLiS6EEEIhZQ1PPQH0l/Rl4FXgMKBlVbGpwFHAY8ChwCzbcRttCCF0IaUcadhuAk4EHgCeT4v8rKRzJR2Um90A9Ja0ADgFGFdg1dd2SMC1KXLRLHLRLHLRLHLRbL1zUeul0UMIIZSo9kujhxBCKE10GiGEEAqridpTHVWCpBYVyMUpwGigCXgLOMb2S6UHWoJquahodyhwFzDM9pMlhliaIrmQJGA86VL2ubZbn+KuxhX4H+lHupG4V24zzvb9pQfawSRNAg4A3rS9ayuvN5DytD/wPnC07dYnQ6/Q5Y80OrIESa0pmIu/A0NtDyLdWX9JuVGWo2AukLQl8HPg8XIjLE+RXEjqD/wK2MP2LsBJpQdagoKfizNIF+MMJl3JeWW5UZbmJmDfdl7fD+ifH8cCVxVZaZfvNOjAEiQ1qGoubD9o+/38dA7pnph6VORzAXAeqeNcWWZwJSuSizHARNvLAGy/WXKMZSmSi4+BrfLPPfn0PWN1wfZDtH+v28HAZNsf254D9JJUtRZTLXQarZUg2a6tNvny3rUlSOpNkVxUGgX8qUMj6jxVcyFpMLC97WllBtYJinwuBgADJD0iaU4ewqlHRXIxHjhS0hLgfuBn5YTW5azr/gSojU6jtSOG9SpBUgcK/56SjgSGApd2aESdp91cSOpGGqo8tbSIOk+Rz8WmpGGI7wAjgesl9erguDpDkVyMBG6y3Zc0nn9L/rxsbNZrv1kLiYoSJM2K5AJJewOnAwfZrtepyqrlYktgV2C2pMXA7sBUSUNLi7A8Rf9H7rX9oe1FwHxSJ1JviuRiFGAA248BPYA+pUTXtRTan7RUC1dPRQmSZlVzkYdkrgH2reNxa6iSC9vLqdgRSJoNjK3Tq6eK/I/cQ/6GLakPabhqYalRlqNILl4GhpNysROp03ir1Ci7hqnAiZLuIFUdX2779Wpv6vJHGh1YgqTmFMzFpcAWwF2SnpY0tZPC7VAFc7FRKJiLB4Clkp4DHgROs720cyLuOAVzcSowRtJcYArpUtO6+5IpaQrpi/RASUskjZJ0vKTjc5P7SV8cFgDXAT8tst4oIxJCCKGwLn+kEUIIoeuITiOEEEJh0WmEEEIoLDqNEEIIhUWnEUIIobDoNELNkXSrpPGdHUc1kuZL2rOd16dLOqLMmEL4rGrh5r5Qp/Kd2l8A1lQsHmC79AJykm4FBKzOjyeBE22/sL7rtD2wYv3nA31tH13x+vfXO+A25IoIH5JKXX8MvEO6F+GXtj8q8P69SeXEv7ShYwv1ITqN0NkOtD2js4PILrA9XtLmpBtGJwHf6uSY1tcuthdLGgA8BDwH3NjJMYU6EJ1G6HJy8TiTdtg9gKeBE2w/30rbbUjzBnwT+AiYZ3uv/Fpf4PK8nhXAb2xPrLZ92+/lu2lvzuvpQSqv/qO8jTtJE/esrrL9JcCRpDv0fwE05Amh5tseIulh4Pq8vjeA3Wz/M793W2AR6ehkab6b+TxgB2AecLzteQV+lxckPQp8vSJno0l3RfcF3gQutH29pJ7AfUCjpBW5+VeA/5CqLIwi1XWbQfp7LKu2/VB/4pxG6KqmkQrqbUvaSd7SRrvTSKUQPp/bngmfTMYzjVSLaDtgH+A0ScOrbThP3HQ4aUIrgLNIFYMHAYOBPUgTGrW5/Uq5NPslwG22t7A9pMXrH9BcG2qtEcDM3GEMI5V5GE0q+T8JuFdS9wK/y0453gUVi98AfkCaU2IMcLmkQble14HAyznOLXL9slNy+71IHc17wO+qbTvUpzjSCJ3tHklN+efZtg/JY+83rW2QT3q/JWlz2++1eP+HwI5AP9svAn/Jy3cHtrJ9QX6+QNINpAJ2M9uIZZykk4APSDP9HZOXHwGMsf1Wjudc0jSZ57Sz/XV1O2lHfHZ+fnjeBqRZ1a60/UR+PknS6cAw4JE21vdM7jg3A24jFbEEwPZ9Fe1mSZoJ7Ak808a6jgNG234VPvl7LJB0VJHzJKG+RKcROtshLc9p5J3dhaSKxX1Iwz7kn1t2GheRdt4zJa0BrrZ9KWkYp5+kdyrabgLMbieWi2yPb2X5F4HKedZfonmymra2v65mkGZOG0I6eb0LcG9+bQfgCEknV7TvTvsT5gwiVXMdAZxP6jxWA0g6gHRE1J802rAZ6YisLf2A+yRVdhAfA9sA/y7yy4X6EZ1G6Ip+TJoc53ukHXRvUunqT00aY/u/wMnAyZK+Bjwo6a+kGcn+ZXunDRDP66Qd9/z8vB+p7Hab27fd8oij3cqgtpsk3UUaolpOmvtibQf5CnCO7YvXJeh8FDBF0iGkebHHSvocaUrkw4A/2v5Q0jSac9tanEuAw23X7TzrobjoNEJXtCWwClhK+hb867YaSjqQdGXQQtLOdk1+zAFWSzoVmEgaRtoZ6G77qXWMZwpwlqS/kXauZwK3Vtl+S28Ae0pqaKcM9+2kOa1XAGMrll9LKnU/i3Qp8ObAd0nzxrQ88mrNhcDDki4m5aE7qRNek486huf1ro2zj6Qtbb+bl10NXCDpJ7Zfzif/d7ddl2X3Q/viRHjoim4kzSD2GvAs8Gg7bQcCs0g72keAy2w/nOdV2B/YDVhMugLoGtLJ33V1DjAX+Adp3P9x0o64ze23so47STvrt/ORUGseBZpIJ9Wnr12Yv+GfAFwFLANeIF2VVYjtp0nzKoy1/Q7pyOgPpNktDyVdMLC27TzgbmCxpHdyBzEB+DNpCO7dHOewotsP9SXm0wghhFBYHGmEEEIoLDqNEEIIhUWnEUIIobDoNEIIIRQWnUYIIYTCotMIIYRQWHQaIYQQCotOI4QQQmH/A+nE9wrNrHjhAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# 5.ROC曲线 decision_function\n", + "### 像其他scikit-learn分类器一样,它StackingCVClassifier具有decision_function可用于绘制ROC曲线的方法。\n", + "### 请注意,decision_function期望并要求元分类器实现decision_function。\n", + "from sklearn import model_selection\n", + "from sklearn.linear_model import LogisticRegression\n", + "from sklearn.neighbors import KNeighborsClassifier\n", + "from sklearn.svm import SVC\n", + "from sklearn.ensemble import RandomForestClassifier\n", + "from mlxtend.classifier import StackingCVClassifier\n", + "from sklearn.metrics import roc_curve, auc\n", + "from sklearn.model_selection import train_test_split\n", + "from sklearn import datasets\n", + "from sklearn.preprocessing import label_binarize\n", + "from sklearn.multiclass import OneVsRestClassifier\n", + "\n", + "iris = datasets.load_iris()\n", + "X, y = iris.data[:, [0, 1]], iris.target\n", + "\n", + "# Binarize the output\n", + "y = label_binarize(y, classes=[0, 1, 2])\n", + "n_classes = y.shape[1]\n", + "\n", + "RANDOM_SEED = 42\n", + "\n", + "X_train, X_test, y_train, y_test = train_test_split(\n", + " X, y, test_size=0.33, random_state=RANDOM_SEED)\n", + "\n", + "clf1 = LogisticRegression()\n", + "clf2 = RandomForestClassifier(random_state=RANDOM_SEED)\n", + "clf3 = SVC(random_state=RANDOM_SEED)\n", + "lr = LogisticRegression()\n", + "\n", + "sclf = StackingCVClassifier(classifiers=[clf1, clf2, clf3],\n", + " meta_classifier=lr)\n", + "\n", + "# Learn to predict each class against the other\n", + "classifier = OneVsRestClassifier(sclf)\n", + "y_score = classifier.fit(X_train, y_train).decision_function(X_test)\n", + "\n", + "# Compute ROC curve and ROC area for each class\n", + "fpr = dict()\n", + "tpr = dict()\n", + "roc_auc = dict()\n", + "for i in range(n_classes):\n", + " fpr[i], tpr[i], _ = roc_curve(y_test[:, i], y_score[:, i])\n", + " roc_auc[i] = auc(fpr[i], tpr[i])\n", + "\n", + "# Compute micro-average ROC curve and ROC area\n", + "fpr[\"micro\"], tpr[\"micro\"], _ = roc_curve(y_test.ravel(), y_score.ravel())\n", + "roc_auc[\"micro\"] = auc(fpr[\"micro\"], tpr[\"micro\"])\n", + "\n", + "plt.figure()\n", + "lw = 2\n", + "plt.plot(fpr[2], tpr[2], color='darkorange',\n", + " lw=lw, label='ROC curve (area = %0.2f)' % roc_auc[2])\n", + "plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n", + "plt.xlim([0.0, 1.0])\n", + "plt.ylim([0.0, 1.05])\n", + "plt.xlabel('False Positive Rate')\n", + "plt.ylabel('True Positive Rate')\n", + "plt.title('Receiver operating characteristic example')\n", + "plt.legend(loc=\"lower right\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Blending与Stacking对比:** \n", + "Blending的优点在于:\n", + " - 比stacking简单(因为不用进行k次的交叉验证来获得stacker feature)\n", + "\n", + "而缺点在于:\n", + " - 使用了很少的数据(是划分hold-out作为测试集,并非cv)\n", + " - blender可能会过拟合(其实大概率是第一点导致的)\n", + " - stacking使用多次的CV会比较稳健" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 4. 结语" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "在本章中,我们讨论了如何使用Blending和Stacking的方式去集成多个模型,相比于Bagging与Boosting的集成方式,Blending和Stacking的方式更加简单和直观,且效果还很好,因此在比赛中有这么一句话:它(Stacking)可以帮你打败当前学术界性能最好的算法  。那么截至目前为止,我们已经把所有的集成学习方式都讨论完了,接下来的第六章,我们将以几个大型的案例来展示集成学习的威力。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}