산탄데르 고객 만족 예측 - 분류

학습 내용

• 캐글의 산탄데르 고객 만족 데이터 세트에 대해 고객 만족 여부를 XGBoost와 LightGBM을 활용하여 예측

대회 설명

• URL :https://www.kaggle.com/competitions/santander-customer-satisfaction/overview
• 어떤 고객이 행복한 고객입니까? 이를 예측하는 대회
• 평가지표 : AUC - ROC-AUC(ROC 곡선 영역)

데이터 설명

• 데이터 다운로드 : https://www.kaggle.com/c/santander-customer-satisfaction/data

  • train(59MB) : target를 포함한 데이터 셋
  • test(59MB) : target이 없는 데이터 셋
  • sample_submission : 제출용 데이터

• 370개의 피처로 이루어진 데이터

• 피처 이름은 전부 익명처리되어 있음.
• 클래스 레이블 명은 TARGET
  • 값이 1이면 불만족 고객.
  • 값이 0이면 만족 고객

데이터 로드 및 전처리

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib

train = pd.read_csv("../../dataset/santander_customer/train.csv", encoding='latin-1')
test = pd.read_csv("../../dataset/santander_customer/test.csv", encoding='latin-1')
sub = pd.read_csv("../../dataset/santander_customer/sample_submission.csv")

train.shape, test.shape, sub.shape

((76020, 371), (75818, 370), (75818, 2))

train.head()

	ID	var3	var15	imp_ent_var16_ult1	imp_op_var39_comer_ult1	imp_op_var39_comer_ult3	imp_op_var40_comer_ult1	imp_op_var40_comer_ult3	imp_op_var40_efect_ult1	imp_op_var40_efect_ult3	...	saldo_medio_var33_hace2	saldo_medio_var33_hace3	saldo_medio_var33_ult1	saldo_medio_var33_ult3	saldo_medio_var44_hace2	saldo_medio_var44_hace3	saldo_medio_var44_ult1	saldo_medio_var44_ult3	var38	TARGET
0	1	2	23	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	39205.170000	0
1	3	2	34	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	49278.030000	0
2	4	2	23	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	67333.770000	0
3	8	2	37	0.0	195.0	195.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	64007.970000	0
4	10	2	39	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	117310.979016	0

5 rows × 371 columns

train.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 76020 entries, 0 to 76019
Columns: 371 entries, ID to TARGET
dtypes: float64(111), int64(260)
memory usage: 215.2 MB

• 111개의 피처가 float형,
• 260개의 피처가 int형
• 모든 피처가 숫자형이며
• NUll값은 없다.

cnt=0
for one in train.columns:
    print(one, end="  ")
    cnt += 1
    if cnt%20==0:
        print()

ID  var3  var15  imp_ent_var16_ult1  imp_op_var39_comer_ult1  imp_op_var39_comer_ult3  imp_op_var40_comer_ult1  imp_op_var40_comer_ult3  imp_op_var40_efect_ult1  imp_op_var40_efect_ult3  imp_op_var40_ult1  imp_op_var41_comer_ult1  imp_op_var41_comer_ult3  imp_op_var41_efect_ult1  imp_op_var41_efect_ult3  imp_op_var41_ult1  imp_op_var39_efect_ult1  imp_op_var39_efect_ult3  imp_op_var39_ult1  imp_sal_var16_ult1  
ind_var1_0  ind_var1  ind_var2_0  ind_var2  ind_var5_0  ind_var5  ind_var6_0  ind_var6  ind_var8_0  ind_var8  ind_var12_0  ind_var12  ind_var13_0  ind_var13_corto_0  ind_var13_corto  ind_var13_largo_0  ind_var13_largo  ind_var13_medio_0  ind_var13_medio  ind_var13  
ind_var14_0  ind_var14  ind_var17_0  ind_var17  ind_var18_0  ind_var18  ind_var19  ind_var20_0  ind_var20  ind_var24_0  ind_var24  ind_var25_cte  ind_var26_0  ind_var26_cte  ind_var26  ind_var25_0  ind_var25  ind_var27_0  ind_var28_0  ind_var28  
ind_var27  ind_var29_0  ind_var29  ind_var30_0  ind_var30  ind_var31_0  ind_var31  ind_var32_cte  ind_var32_0  ind_var32  ind_var33_0  ind_var33  ind_var34_0  ind_var34  ind_var37_cte  ind_var37_0  ind_var37  ind_var39_0  ind_var40_0  ind_var40  
ind_var41_0  ind_var41  ind_var39  ind_var44_0  ind_var44  ind_var46_0  ind_var46  num_var1_0  num_var1  num_var4  num_var5_0  num_var5  num_var6_0  num_var6  num_var8_0  num_var8  num_var12_0  num_var12  num_var13_0  num_var13_corto_0  
num_var13_corto  num_var13_largo_0  num_var13_largo  num_var13_medio_0  num_var13_medio  num_var13  num_var14_0  num_var14  num_var17_0  num_var17  num_var18_0  num_var18  num_var20_0  num_var20  num_var24_0  num_var24  num_var26_0  num_var26  num_var25_0  num_var25  
num_op_var40_hace2  num_op_var40_hace3  num_op_var40_ult1  num_op_var40_ult3  num_op_var41_hace2  num_op_var41_hace3  num_op_var41_ult1  num_op_var41_ult3  num_op_var39_hace2  num_op_var39_hace3  num_op_var39_ult1  num_op_var39_ult3  num_var27_0  num_var28_0  num_var28  num_var27  num_var29_0  num_var29  num_var30_0  num_var30  
num_var31_0  num_var31  num_var32_0  num_var32  num_var33_0  num_var33  num_var34_0  num_var34  num_var35  num_var37_med_ult2  num_var37_0  num_var37  num_var39_0  num_var40_0  num_var40  num_var41_0  num_var41  num_var39  num_var42_0  num_var42  
num_var44_0  num_var44  num_var46_0  num_var46  saldo_var1  saldo_var5  saldo_var6  saldo_var8  saldo_var12  saldo_var13_corto  saldo_var13_largo  saldo_var13_medio  saldo_var13  saldo_var14  saldo_var17  saldo_var18  saldo_var20  saldo_var24  saldo_var26  saldo_var25  
saldo_var28  saldo_var27  saldo_var29  saldo_var30  saldo_var31  saldo_var32  saldo_var33  saldo_var34  saldo_var37  saldo_var40  saldo_var41  saldo_var42  saldo_var44  saldo_var46  var36  delta_imp_amort_var18_1y3  delta_imp_amort_var34_1y3  delta_imp_aport_var13_1y3  delta_imp_aport_var17_1y3  delta_imp_aport_var33_1y3  
delta_imp_compra_var44_1y3  delta_imp_reemb_var13_1y3  delta_imp_reemb_var17_1y3  delta_imp_reemb_var33_1y3  delta_imp_trasp_var17_in_1y3  delta_imp_trasp_var17_out_1y3  delta_imp_trasp_var33_in_1y3  delta_imp_trasp_var33_out_1y3  delta_imp_venta_var44_1y3  delta_num_aport_var13_1y3  delta_num_aport_var17_1y3  delta_num_aport_var33_1y3  delta_num_compra_var44_1y3  delta_num_reemb_var13_1y3  delta_num_reemb_var17_1y3  delta_num_reemb_var33_1y3  delta_num_trasp_var17_in_1y3  delta_num_trasp_var17_out_1y3  delta_num_trasp_var33_in_1y3  delta_num_trasp_var33_out_1y3  
delta_num_venta_var44_1y3  imp_amort_var18_hace3  imp_amort_var18_ult1  imp_amort_var34_hace3  imp_amort_var34_ult1  imp_aport_var13_hace3  imp_aport_var13_ult1  imp_aport_var17_hace3  imp_aport_var17_ult1  imp_aport_var33_hace3  imp_aport_var33_ult1  imp_var7_emit_ult1  imp_var7_recib_ult1  imp_compra_var44_hace3  imp_compra_var44_ult1  imp_reemb_var13_hace3  imp_reemb_var13_ult1  imp_reemb_var17_hace3  imp_reemb_var17_ult1  imp_reemb_var33_hace3  
imp_reemb_var33_ult1  imp_var43_emit_ult1  imp_trans_var37_ult1  imp_trasp_var17_in_hace3  imp_trasp_var17_in_ult1  imp_trasp_var17_out_hace3  imp_trasp_var17_out_ult1  imp_trasp_var33_in_hace3  imp_trasp_var33_in_ult1  imp_trasp_var33_out_hace3  imp_trasp_var33_out_ult1  imp_venta_var44_hace3  imp_venta_var44_ult1  ind_var7_emit_ult1  ind_var7_recib_ult1  ind_var10_ult1  ind_var10cte_ult1  ind_var9_cte_ult1  ind_var9_ult1  ind_var43_emit_ult1  
ind_var43_recib_ult1  var21  num_var2_0_ult1  num_var2_ult1  num_aport_var13_hace3  num_aport_var13_ult1  num_aport_var17_hace3  num_aport_var17_ult1  num_aport_var33_hace3  num_aport_var33_ult1  num_var7_emit_ult1  num_var7_recib_ult1  num_compra_var44_hace3  num_compra_var44_ult1  num_ent_var16_ult1  num_var22_hace2  num_var22_hace3  num_var22_ult1  num_var22_ult3  num_med_var22_ult3  
num_med_var45_ult3  num_meses_var5_ult3  num_meses_var8_ult3  num_meses_var12_ult3  num_meses_var13_corto_ult3  num_meses_var13_largo_ult3  num_meses_var13_medio_ult3  num_meses_var17_ult3  num_meses_var29_ult3  num_meses_var33_ult3  num_meses_var39_vig_ult3  num_meses_var44_ult3  num_op_var39_comer_ult1  num_op_var39_comer_ult3  num_op_var40_comer_ult1  num_op_var40_comer_ult3  num_op_var40_efect_ult1  num_op_var40_efect_ult3  num_op_var41_comer_ult1  num_op_var41_comer_ult3  
num_op_var41_efect_ult1  num_op_var41_efect_ult3  num_op_var39_efect_ult1  num_op_var39_efect_ult3  num_reemb_var13_hace3  num_reemb_var13_ult1  num_reemb_var17_hace3  num_reemb_var17_ult1  num_reemb_var33_hace3  num_reemb_var33_ult1  num_sal_var16_ult1  num_var43_emit_ult1  num_var43_recib_ult1  num_trasp_var11_ult1  num_trasp_var17_in_hace3  num_trasp_var17_in_ult1  num_trasp_var17_out_hace3  num_trasp_var17_out_ult1  num_trasp_var33_in_hace3  num_trasp_var33_in_ult1  
num_trasp_var33_out_hace3  num_trasp_var33_out_ult1  num_venta_var44_hace3  num_venta_var44_ult1  num_var45_hace2  num_var45_hace3  num_var45_ult1  num_var45_ult3  saldo_var2_ult1  saldo_medio_var5_hace2  saldo_medio_var5_hace3  saldo_medio_var5_ult1  saldo_medio_var5_ult3  saldo_medio_var8_hace2  saldo_medio_var8_hace3  saldo_medio_var8_ult1  saldo_medio_var8_ult3  saldo_medio_var12_hace2  saldo_medio_var12_hace3  saldo_medio_var12_ult1  
saldo_medio_var12_ult3  saldo_medio_var13_corto_hace2  saldo_medio_var13_corto_hace3  saldo_medio_var13_corto_ult1  saldo_medio_var13_corto_ult3  saldo_medio_var13_largo_hace2  saldo_medio_var13_largo_hace3  saldo_medio_var13_largo_ult1  saldo_medio_var13_largo_ult3  saldo_medio_var13_medio_hace2  saldo_medio_var13_medio_hace3  saldo_medio_var13_medio_ult1  saldo_medio_var13_medio_ult3  saldo_medio_var17_hace2  saldo_medio_var17_hace3  saldo_medio_var17_ult1  saldo_medio_var17_ult3  saldo_medio_var29_hace2  saldo_medio_var29_hace3  saldo_medio_var29_ult1  
saldo_medio_var29_ult3  saldo_medio_var33_hace2  saldo_medio_var33_hace3  saldo_medio_var33_ult1  saldo_medio_var33_ult3  saldo_medio_var44_hace2  saldo_medio_var44_hace3  saldo_medio_var44_ult1  saldo_medio_var44_ult3  var38  TARGET  

전체 데이터의 만족(0), 불만족(1) 비율

train['TARGET'].value_counts()

0    73012
1     3008
Name: TARGET, dtype: int64

train['TARGET'].value_counts()[0]

73012

satified = train['TARGET'].value_counts()[0]    # 만족
unsatified = train['TARGET'].value_counts()[1]  # 불만족
all_count = train['TARGET'].count()

print("{:.3f}% {:.3f}%".format( (satified/all_count) * 100 , 
                               (unsatified/all_count) * 100 ) )

96.043% 3.957%

train.describe()

	ID	var3	var15	imp_ent_var16_ult1	imp_op_var39_comer_ult1	imp_op_var39_comer_ult3	imp_op_var40_comer_ult1	imp_op_var40_comer_ult3	imp_op_var40_efect_ult1	imp_op_var40_efect_ult3	...	saldo_medio_var33_hace2	saldo_medio_var33_hace3	saldo_medio_var33_ult1	saldo_medio_var33_ult3	saldo_medio_var44_hace2	saldo_medio_var44_hace3	saldo_medio_var44_ult1	saldo_medio_var44_ult3	var38	TARGET
count	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	...	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	76020.000000	7.602000e+04	76020.000000
mean	75964.050723	-1523.199277	33.212865	86.208265	72.363067	119.529632	3.559130	6.472698	0.412946	0.567352	...	7.935824	1.365146	12.215580	8.784074	31.505324	1.858575	76.026165	56.614351	1.172358e+05	0.039569
std	43781.947379	39033.462364	12.956486	1614.757313	339.315831	546.266294	93.155749	153.737066	30.604864	36.513513	...	455.887218	113.959637	783.207399	538.439211	2013.125393	147.786584	4040.337842	2852.579397	1.826646e+05	0.194945
min	1.000000	-999999.000000	5.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	5.163750e+03	0.000000
25%	38104.750000	2.000000	23.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	6.787061e+04	0.000000
50%	76043.000000	2.000000	28.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.064092e+05	0.000000
75%	113748.750000	2.000000	40.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	1.187563e+05	0.000000
max	151838.000000	238.000000	105.000000	210000.000000	12888.030000	21024.810000	8237.820000	11073.570000	6600.000000	6600.000000	...	50003.880000	20385.720000	138831.630000	91778.730000	438329.220000	24650.010000	681462.900000	397884.300000	2.203474e+07	1.000000

8 rows × 371 columns

• var3의 최소값이 -999999 - 이상치로 보임

train['var3'].value_counts()

 2         74165
 8           138
-999999      116
 9           110
 3           108
           ...  
 177           1
 87            1
 151           1
 215           1
 191           1
Name: var3, Length: 208, dtype: int64

# -999999를 가장 많은 나온 값으로 변경
train['var3'].replace(-999999, 2, inplace=True)

# 실제 확인
train.loc[ train['var3']==-999999, : ]

	ID	var3	var15	imp_ent_var16_ult1	imp_op_var39_comer_ult1	imp_op_var39_comer_ult3	imp_op_var40_comer_ult1	imp_op_var40_comer_ult3	imp_op_var40_efect_ult1	imp_op_var40_efect_ult3	...	saldo_medio_var33_hace2	saldo_medio_var33_hace3	saldo_medio_var33_ult1	saldo_medio_var33_ult3	saldo_medio_var44_hace2	saldo_medio_var44_hace3	saldo_medio_var44_ult1	saldo_medio_var44_ult3	var38	TARGET

0 rows × 371 columns

## ID 열을 삭제
# train.drop('ID', axis=1, inplace=True)
train = train.loc[  :, "var3":  ]
train.head()

	var3	var15	imp_ent_var16_ult1	imp_op_var39_comer_ult1	imp_op_var39_comer_ult3	imp_op_var40_comer_ult1	imp_op_var40_comer_ult3	imp_op_var40_efect_ult1	imp_op_var40_efect_ult3	imp_op_var40_ult1	...	saldo_medio_var33_hace2	saldo_medio_var33_hace3	saldo_medio_var33_ult1	saldo_medio_var33_ult3	saldo_medio_var44_hace2	saldo_medio_var44_hace3	saldo_medio_var44_ult1	saldo_medio_var44_ult3	var38	TARGET
0	2	23	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	39205.170000	0
1	2	34	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	49278.030000	0
2	2	23	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	67333.770000	0
3	2	37	0.0	195.0	195.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	64007.970000	0
4	2	39	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	...	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	117310.979016	0

5 rows × 370 columns

# 피처와 레이블를 지정.
# TARGET를 제외한 열을 입력으로(X), TARGET열을 y로 지정
X = train.iloc[:, :-1]
y = train['TARGET']

X.shape, y.shape

((76020, 369), (76020,))

데이터 나누기

• 학습용 80%, 자체 검증용 20%

from sklearn.model_selection import train_test_split

X_train , X_test, y_train, y_test = train_test_split(X, y, 
                                                     stratify=y,
                                                     test_size=0.2, random_state=0)

X_train.shape, X_test.shape, y_train.shape, y_test.shape

((60816, 369), (15204, 369), (60816,), (15204,))

y_test[0:10]

19379    0
66921    0
12415    0
9735     0
17997    0
67089    0
63376    0
6461     0
33577    0
59255    0
Name: TARGET, dtype: int64

## target(레이블) 분포비율
print( "학습용 레이블 분포 비율 : \n" , y_train.value_counts() / y_train.count() )
print( "테스트용 레이블 분포 비율 : \n" , y_train.value_counts() / y_train.count() )

학습용 레이블 분포 비율 : 
 0    0.960438
1    0.039562
Name: TARGET, dtype: float64
테스트용 레이블 분포 비율 : 
 0    0.960438
1    0.039562
Name: TARGET, dtype: float64

모델 생성 및 학습, 그리고 평가해 보기

%%time

from xgboost import XGBClassifier
from sklearn.metrics import roc_auc_score

xgb_model = XGBClassifier(n_estimators=500, random_state=156)
xgb_model.fit(X_train, y_train, 
              early_stopping_rounds=100,
              eval_metric='auc', 
              eval_set=[(X_train, y_train), (X_test, y_test)])

[0]	validation_0-auc:0.82570	validation_1-auc:0.79283
[1]	validation_0-auc:0.84010	validation_1-auc:0.80737
[2]	validation_0-auc:0.84361	validation_1-auc:0.81021
[3]	validation_0-auc:0.84783	validation_1-auc:0.81287
[4]	validation_0-auc:0.85123	validation_1-auc:0.81469
[5]	validation_0-auc:0.85518	validation_1-auc:0.81860
[6]	validation_0-auc:0.85922	validation_1-auc:0.81977
[7]	validation_0-auc:0.86238	validation_1-auc:0.82034
[8]	validation_0-auc:0.86570	validation_1-auc:0.82147
[9]	validation_0-auc:0.86798	validation_1-auc:0.82301
[10]	validation_0-auc:0.87104	validation_1-auc:0.82379
[11]	validation_0-auc:0.87448	validation_1-auc:0.82456
[12]	validation_0-auc:0.87687	validation_1-auc:0.82401
[13]	validation_0-auc:0.87918	validation_1-auc:0.82467
[14]	validation_0-auc:0.88081	validation_1-auc:0.82508
[15]	validation_0-auc:0.88331	validation_1-auc:0.82379
[16]	validation_0-auc:0.88569	validation_1-auc:0.82457
[17]	validation_0-auc:0.88674	validation_1-auc:0.82453
[18]	validation_0-auc:0.88885	validation_1-auc:0.82354
[19]	validation_0-auc:0.89038	validation_1-auc:0.82328
[20]	validation_0-auc:0.89090	validation_1-auc:0.82305
[21]	validation_0-auc:0.89327	validation_1-auc:0.82305
[22]	validation_0-auc:0.89397	validation_1-auc:0.82287
[23]	validation_0-auc:0.89712	validation_1-auc:0.82256
[24]	validation_0-auc:0.89745	validation_1-auc:0.82237
[25]	validation_0-auc:0.89779	validation_1-auc:0.82237
[26]	validation_0-auc:0.89844	validation_1-auc:0.82248
[27]	validation_0-auc:0.89912	validation_1-auc:0.82199
[28]	validation_0-auc:0.89987	validation_1-auc:0.82157
[29]	validation_0-auc:0.90071	validation_1-auc:0.82106
[30]	validation_0-auc:0.90210	validation_1-auc:0.82075
[31]	validation_0-auc:0.90433	validation_1-auc:0.82011
[32]	validation_0-auc:0.90495	validation_1-auc:0.82001
[33]	validation_0-auc:0.90548	validation_1-auc:0.82032
[34]	validation_0-auc:0.90610	validation_1-auc:0.81969
[35]	validation_0-auc:0.90661	validation_1-auc:0.81968
[36]	validation_0-auc:0.90682	validation_1-auc:0.81939
[37]	validation_0-auc:0.90842	validation_1-auc:0.81969
[38]	validation_0-auc:0.90869	validation_1-auc:0.81962
[39]	validation_0-auc:0.90888	validation_1-auc:0.81957
[40]	validation_0-auc:0.91045	validation_1-auc:0.81893
[41]	validation_0-auc:0.91171	validation_1-auc:0.81869
[42]	validation_0-auc:0.91239	validation_1-auc:0.81802
[43]	validation_0-auc:0.91329	validation_1-auc:0.81773
[44]	validation_0-auc:0.91353	validation_1-auc:0.81771
[45]	validation_0-auc:0.91424	validation_1-auc:0.81696
[46]	validation_0-auc:0.91443	validation_1-auc:0.81668
[47]	validation_0-auc:0.91459	validation_1-auc:0.81649
[48]	validation_0-auc:0.91483	validation_1-auc:0.81643
[49]	validation_0-auc:0.91501	validation_1-auc:0.81639
[50]	validation_0-auc:0.91534	validation_1-auc:0.81613
[51]	validation_0-auc:0.91557	validation_1-auc:0.81593
[52]	validation_0-auc:0.91582	validation_1-auc:0.81550
[53]	validation_0-auc:0.91601	validation_1-auc:0.81585
[54]	validation_0-auc:0.91722	validation_1-auc:0.81637
[55]	validation_0-auc:0.91873	validation_1-auc:0.81594
[56]	validation_0-auc:0.91876	validation_1-auc:0.81572
[57]	validation_0-auc:0.91937	validation_1-auc:0.81568
[58]	validation_0-auc:0.91948	validation_1-auc:0.81541
[59]	validation_0-auc:0.91974	validation_1-auc:0.81575
[60]	validation_0-auc:0.92002	validation_1-auc:0.81554
[61]	validation_0-auc:0.92174	validation_1-auc:0.81550
[62]	validation_0-auc:0.92218	validation_1-auc:0.81531
[63]	validation_0-auc:0.92270	validation_1-auc:0.81534
[64]	validation_0-auc:0.92359	validation_1-auc:0.81497
[65]	validation_0-auc:0.92370	validation_1-auc:0.81487
[66]	validation_0-auc:0.92402	validation_1-auc:0.81483
[67]	validation_0-auc:0.92411	validation_1-auc:0.81483
[68]	validation_0-auc:0.92477	validation_1-auc:0.81450
[69]	validation_0-auc:0.92482	validation_1-auc:0.81449
[70]	validation_0-auc:0.92492	validation_1-auc:0.81436
[71]	validation_0-auc:0.92502	validation_1-auc:0.81401
[72]	validation_0-auc:0.92566	validation_1-auc:0.81426
[73]	validation_0-auc:0.92575	validation_1-auc:0.81447
[74]	validation_0-auc:0.92588	validation_1-auc:0.81450
[75]	validation_0-auc:0.92600	validation_1-auc:0.81438
[76]	validation_0-auc:0.92606	validation_1-auc:0.81431
[77]	validation_0-auc:0.92617	validation_1-auc:0.81414
[78]	validation_0-auc:0.92731	validation_1-auc:0.81417
[79]	validation_0-auc:0.92737	validation_1-auc:0.81399
[80]	validation_0-auc:0.92810	validation_1-auc:0.81361
[81]	validation_0-auc:0.92852	validation_1-auc:0.81289
[82]	validation_0-auc:0.92888	validation_1-auc:0.81225
[83]	validation_0-auc:0.92964	validation_1-auc:0.81179
[84]	validation_0-auc:0.93017	validation_1-auc:0.81139
[85]	validation_0-auc:0.93041	validation_1-auc:0.81168
[86]	validation_0-auc:0.93056	validation_1-auc:0.81134
[87]	validation_0-auc:0.93103	validation_1-auc:0.81192
[88]	validation_0-auc:0.93154	validation_1-auc:0.81169
[89]	validation_0-auc:0.93212	validation_1-auc:0.81172
[90]	validation_0-auc:0.93268	validation_1-auc:0.81083
[91]	validation_0-auc:0.93397	validation_1-auc:0.81032
[92]	validation_0-auc:0.93411	validation_1-auc:0.81025
[93]	validation_0-auc:0.93418	validation_1-auc:0.81013
[94]	validation_0-auc:0.93428	validation_1-auc:0.80974
[95]	validation_0-auc:0.93481	validation_1-auc:0.80971
[96]	validation_0-auc:0.93514	validation_1-auc:0.80948
[97]	validation_0-auc:0.93518	validation_1-auc:0.80927
[98]	validation_0-auc:0.93539	validation_1-auc:0.80908
[99]	validation_0-auc:0.93546	validation_1-auc:0.80912
[100]	validation_0-auc:0.93575	validation_1-auc:0.80906
[101]	validation_0-auc:0.93621	validation_1-auc:0.80839
[102]	validation_0-auc:0.93663	validation_1-auc:0.80831
[103]	validation_0-auc:0.93679	validation_1-auc:0.80823
[104]	validation_0-auc:0.93684	validation_1-auc:0.80824
[105]	validation_0-auc:0.93701	validation_1-auc:0.80766
[106]	validation_0-auc:0.93711	validation_1-auc:0.80736
[107]	validation_0-auc:0.93712	validation_1-auc:0.80722
[108]	validation_0-auc:0.93715	validation_1-auc:0.80720
[109]	validation_0-auc:0.93836	validation_1-auc:0.80643
[110]	validation_0-auc:0.93912	validation_1-auc:0.80620
[111]	validation_0-auc:0.93930	validation_1-auc:0.80622
[112]	validation_0-auc:0.93933	validation_1-auc:0.80625
[113]	validation_0-auc:0.93939	validation_1-auc:0.80600
CPU times: user 6min 42s, sys: 1.35 s, total: 6min 43s
Wall time: 1min 50s

XGBClassifier(base_score=0.5, booster='gbtree', callbacks=None,
              colsample_bylevel=1, colsample_bynode=1, colsample_bytree=1,
              early_stopping_rounds=None, enable_categorical=False,
              eval_metric=None, gamma=0, gpu_id=-1, grow_policy='depthwise',
              importance_type=None, interaction_constraints='',
              learning_rate=0.300000012, max_bin=256, max_cat_to_onehot=4,
              max_delta_step=0, max_depth=6, max_leaves=0, min_child_weight=1,
              missing=nan, monotone_constraints='()', n_estimators=500,
              n_jobs=0, num_parallel_tree=1, predictor='auto', random_state=156,
              reg_alpha=0, reg_lambda=1, ...)

# 0의 예측 확률, 1의 예측 확률
pred_prob = xgb_model.predict_proba(X_test)[:, 1]
pred_prob

array([0.00690398, 0.02649283, 0.01910355, ..., 0.01988643, 0.01178615,
       0.00611465], dtype=float32)

# 실제값(y_test)와 예측값(pred_prob)
xgb_roc_score = roc_auc_score(y_test, pred_prob, average='macro')
print("ROC AUC : {0:.4f}".format(xgb_roc_score))

ROC AUC : 0.8251

임계값을 정해서 평가해보기

pred_01 = pred_prob > 0.1
pred_01

array([False, False, False, ..., False, False, False])

# 실제값(y_test)와 예측값(pred_prob)
xgb_roc_score = roc_auc_score(y_test, pred_01, average='macro')
print("ROC AUC : {0:.4f}".format(xgb_roc_score))

ROC AUC : 0.7161

GridSearchCV를 이용한 하이퍼 파라미터 튜닝

• max_depth, min_child_weight, colsample_bytree
• 먼저 2-3개 정도의 파라미터를 최적화 시킨 후,최적 파라미터를 기반으로 1-2개 파라미터를 결합하여 튜닝을 수행

%%time

from sklearn.model_selection import GridSearchCV

# 우선 하이퍼 파라미터 수행 속도를 향상을 위해 100으로
xgb_model1 = XGBClassifier(n_estimators=100, use_label_encoder=False)
params = {"max_depth":[5,7],
          "min_child_weight":[1,3],
          "colsample_bytree":[0.5, 0.75]}

gridcv = GridSearchCV(xgb_model1, param_grid=params, cv=3)
gridcv.fit(X_train, y_train, early_stopping_rounds=30, 
            eval_metric='auc', 
            eval_set = [(X_train, y_train), (X_test, y_test)])

/Users/toto/Documents/anaconda3/lib/python3.8/site-packages/xgboost/sklearn.py:793: UserWarning: `eval_metric` in `fit` method is deprecated for better compatibility with scikit-learn, use `eval_metric` in constructor or`set_params` instead.
  warnings.warn(
/Users/toto/Documents/anaconda3/lib/python3.8/site-packages/xgboost/sklearn.py:793: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(

[0]	validation_0-auc:0.70654	validation_1-auc:0.67817
[1]	validation_0-auc:0.73189	validation_1-auc:0.70514
[2]	validation_0-auc:0.81996	validation_1-auc:0.78750
[3]	validation_0-auc:0.83320	validation_1-auc:0.80213
[4]	validation_0-auc:0.83949	validation_1-auc:0.80794
[5]	validation_0-auc:0.84186	validation_1-auc:0.81103
[6]	validation_0-auc:0.84286	validation_1-auc:0.80874
[7]	validation_0-auc:0.84853	validation_1-auc:0.81403
[8]	validation_0-auc:0.84752	validation_1-auc:0.81134
[9]	validation_0-auc:0.84584	validation_1-auc:0.80799
[10]	validation_0-auc:0.85212	validation_1-auc:0.81358
[11]	validation_0-auc:0.85540	validation_1-auc:0.81618
[12]	validation_0-auc:0.85749	validation_1-auc:0.81738
[13]	validation_0-auc:0.85850	validation_1-auc:0.81641
[14]	validation_0-auc:0.85966	validation_1-auc:0.81634
[15]	validation_0-auc:0.86234	validation_1-auc:0.81877
[16]	validation_0-auc:0.86273	validation_1-auc:0.81800
[17]	validation_0-auc:0.86425	validation_1-auc:0.81967
[18]	validation_0-auc:0.86564	validation_1-auc:0.82098
[19]	validation_0-auc:0.86668	validation_1-auc:0.82119
[20]	validation_0-auc:0.86761	validation_1-auc:0.82160
[21]	validation_0-auc:0.86891	validation_1-auc:0.82274
[22]	validation_0-auc:0.86957	validation_1-auc:0.82306
[23]	validation_0-auc:0.86996	validation_1-auc:0.82292
[24]	validation_0-auc:0.87145	validation_1-auc:0.82130
[25]	validation_0-auc:0.87287	validation_1-auc:0.82121
[26]	validation_0-auc:0.87418	validation_1-auc:0.82145
[27]	validation_0-auc:0.87516	validation_1-auc:0.82088
[28]	validation_0-auc:0.87568	validation_1-auc:0.82098
[29]	validation_0-auc:0.87709	validation_1-auc:0.82103
[30]	validation_0-auc:0.87737	validation_1-auc:0.82146
[31]	validation_0-auc:0.87783	validation_1-auc:0.82152
[32]	validation_0-auc:0.87830	validation_1-auc:0.82142
[33]	validation_0-auc:0.87840	validation_1-auc:0.82115
[34]	validation_0-auc:0.87868	validation_1-auc:0.82108
[35]	validation_0-auc:0.87926	validation_1-auc:0.82019
[36]	validation_0-auc:0.87971	validation_1-auc:0.82024
[37]	validation_0-auc:0.87992	validation_1-auc:0.82014
[38]	validation_0-auc:0.88017	validation_1-auc:0.82000
[39]	validation_0-auc:0.88036	validation_1-auc:0.81942
[40]	validation_0-auc:0.88084	validation_1-auc:0.81915
[41]	validation_0-auc:0.88116	validation_1-auc:0.81877
[42]	validation_0-auc:0.88175	validation_1-auc:0.81840
[43]	validation_0-auc:0.88193	validation_1-auc:0.81775
[44]	validation_0-auc:0.88199	validation_1-auc:0.81749
[45]	validation_0-auc:0.88239	validation_1-auc:0.81748
[46]	validation_0-auc:0.88252	validation_1-auc:0.81770
[47]	validation_0-auc:0.88271	validation_1-auc:0.81790
[48]	validation_0-auc:0.88284	validation_1-auc:0.81772
[49]	validation_0-auc:0.88292	validation_1-auc:0.81753
[50]	validation_0-auc:0.88377	validation_1-auc:0.81666
[51]	validation_0-auc:0.88389	validation_1-auc:0.81628
[0]	validation_0-auc:0.72783	validation_1-auc:0.70300
[1]	validation_0-auc:0.74412	validation_1-auc:0.71808
[2]	validation_0-auc:0.81801	validation_1-auc:0.78752
[3]	validation_0-auc:0.83189	validation_1-auc:0.79897
[4]	validation_0-auc:0.83970	validation_1-auc:0.80782
[5]	validation_0-auc:0.84405	validation_1-auc:0.80935
[6]	validation_0-auc:0.84194	validation_1-auc:0.80586
[7]	validation_0-auc:0.84791	validation_1-auc:0.81407
[8]	validation_0-auc:0.84668	validation_1-auc:0.81057
[9]	validation_0-auc:0.84805	validation_1-auc:0.80636
[10]	validation_0-auc:0.85310	validation_1-auc:0.81102
[11]	validation_0-auc:0.85510	validation_1-auc:0.81332
[12]	validation_0-auc:0.85763	validation_1-auc:0.81515
[13]	validation_0-auc:0.85890	validation_1-auc:0.81460
[14]	validation_0-auc:0.85897	validation_1-auc:0.81161
[15]	validation_0-auc:0.86081	validation_1-auc:0.81430
[16]	validation_0-auc:0.86189	validation_1-auc:0.81387
[17]	validation_0-auc:0.86335	validation_1-auc:0.81461
[18]	validation_0-auc:0.86511	validation_1-auc:0.81774
[19]	validation_0-auc:0.86623	validation_1-auc:0.81740
[20]	validation_0-auc:0.86659	validation_1-auc:0.81752
[21]	validation_0-auc:0.86807	validation_1-auc:0.81909
[22]	validation_0-auc:0.86851	validation_1-auc:0.81930
[23]	validation_0-auc:0.86900	validation_1-auc:0.81904
[24]	validation_0-auc:0.86999	validation_1-auc:0.81830
[25]	validation_0-auc:0.87128	validation_1-auc:0.81845
[26]	validation_0-auc:0.87222	validation_1-auc:0.81886
[27]	validation_0-auc:0.87279	validation_1-auc:0.81824
[28]	validation_0-auc:0.87367	validation_1-auc:0.81821
[29]	validation_0-auc:0.87466	validation_1-auc:0.81844
[30]	validation_0-auc:0.87567	validation_1-auc:0.81902
[31]	validation_0-auc:0.87593	validation_1-auc:0.81873
[32]	validation_0-auc:0.87639	validation_1-auc:0.81877
[33]	validation_0-auc:0.87711	validation_1-auc:0.81870
[34]	validation_0-auc:0.87792	validation_1-auc:0.81827
[35]	validation_0-auc:0.87853	validation_1-auc:0.81799
[36]	validation_0-auc:0.87885	validation_1-auc:0.81808
[37]	validation_0-auc:0.87914	validation_1-auc:0.81773
[38]	validation_0-auc:0.88076	validation_1-auc:0.81908
[39]	validation_0-auc:0.88086	validation_1-auc:0.81922
[40]	validation_0-auc:0.88109	validation_1-auc:0.81929
[41]	validation_0-auc:0.88158	validation_1-auc:0.81939
[42]	validation_0-auc:0.88178	validation_1-auc:0.81911
[43]	validation_0-auc:0.88196	validation_1-auc:0.81909
[44]	validation_0-auc:0.88214	validation_1-auc:0.81890
[45]	validation_0-auc:0.88220	validation_1-auc:0.81874
[46]	validation_0-auc:0.88270	validation_1-auc:0.81810
[47]	validation_0-auc:0.88341	validation_1-auc:0.81796
[48]	validation_0-auc:0.88368	validation_1-auc:0.81781
[49]	validation_0-auc:0.88401	validation_1-auc:0.81762
[50]	validation_0-auc:0.88415	validation_1-auc:0.81761
[51]	validation_0-auc:0.88479	validation_1-auc:0.81745
[52]	validation_0-auc:0.88485	validation_1-auc:0.81712
[53]	validation_0-auc:0.88491	validation_1-auc:0.81681
[54]	validation_0-auc:0.88521	validation_1-auc:0.81616
[55]	validation_0-auc:0.88533	validation_1-auc:0.81624
[56]	validation_0-auc:0.88565	validation_1-auc:0.81593
[57]	validation_0-auc:0.88617	validation_1-auc:0.81595
[58]	validation_0-auc:0.88618	validation_1-auc:0.81562
[59]	validation_0-auc:0.88630	validation_1-auc:0.81560
[60]	validation_0-auc:0.88650	validation_1-auc:0.81493
[61]	validation_0-auc:0.88660	validation_1-auc:0.81476
[62]	validation_0-auc:0.88660	validation_1-auc:0.81468
[63]	validation_0-auc:0.88676	validation_1-auc:0.81467
[64]	validation_0-auc:0.88676	validation_1-auc:0.81451
[65]	validation_0-auc:0.88685	validation_1-auc:0.81469
[66]	validation_0-auc:0.88692	validation_1-auc:0.81456
[67]	validation_0-auc:0.88704	validation_1-auc:0.81431
[68]	validation_0-auc:0.88739	validation_1-auc:0.81398
[69]	validation_0-auc:0.88750	validation_1-auc:0.81384
[70]	validation_0-auc:0.88768	validation_1-auc:0.81384
[0]	validation_0-auc:0.72686	validation_1-auc:0.70676
[1]	validation_0-auc:0.75743	validation_1-auc:0.73285
[2]	validation_0-auc:0.82414	validation_1-auc:0.79894
[3]	validation_0-auc:0.83337	validation_1-auc:0.80629
[4]	validation_0-auc:0.84149	validation_1-auc:0.81352
[5]	validation_0-auc:0.84339	validation_1-auc:0.81542
[6]	validation_0-auc:0.84282	validation_1-auc:0.81244
[7]	validation_0-auc:0.84783	validation_1-auc:0.81590
[8]	validation_0-auc:0.84707	validation_1-auc:0.81315
[9]	validation_0-auc:0.84537	validation_1-auc:0.80735
[10]	validation_0-auc:0.85239	validation_1-auc:0.81294
[11]	validation_0-auc:0.85572	validation_1-auc:0.81638
[12]	validation_0-auc:0.85807	validation_1-auc:0.81984
[13]	validation_0-auc:0.85829	validation_1-auc:0.81772
[14]	validation_0-auc:0.85971	validation_1-auc:0.81629
[15]	validation_0-auc:0.86257	validation_1-auc:0.81841
[16]	validation_0-auc:0.86321	validation_1-auc:0.81860
[17]	validation_0-auc:0.86483	validation_1-auc:0.81997
[18]	validation_0-auc:0.86657	validation_1-auc:0.82091
[19]	validation_0-auc:0.86733	validation_1-auc:0.82025
[20]	validation_0-auc:0.86807	validation_1-auc:0.82080
[21]	validation_0-auc:0.86895	validation_1-auc:0.82040
[22]	validation_0-auc:0.86956	validation_1-auc:0.82003
[23]	validation_0-auc:0.87032	validation_1-auc:0.81916
[24]	validation_0-auc:0.87056	validation_1-auc:0.81930
[25]	validation_0-auc:0.87155	validation_1-auc:0.81935
[26]	validation_0-auc:0.87225	validation_1-auc:0.82001
[27]	validation_0-auc:0.87331	validation_1-auc:0.81986
[28]	validation_0-auc:0.87406	validation_1-auc:0.82011
[29]	validation_0-auc:0.87454	validation_1-auc:0.81943
[30]	validation_0-auc:0.87490	validation_1-auc:0.81913
[31]	validation_0-auc:0.87504	validation_1-auc:0.81875
[32]	validation_0-auc:0.87571	validation_1-auc:0.81834
[33]	validation_0-auc:0.87619	validation_1-auc:0.81888
[34]	validation_0-auc:0.87628	validation_1-auc:0.81893
[35]	validation_0-auc:0.87643	validation_1-auc:0.81903
[36]	validation_0-auc:0.87665	validation_1-auc:0.81859
[37]	validation_0-auc:0.87681	validation_1-auc:0.81822
[38]	validation_0-auc:0.87696	validation_1-auc:0.81810
[39]	validation_0-auc:0.87720	validation_1-auc:0.81735
[40]	validation_0-auc:0.87737	validation_1-auc:0.81736
[41]	validation_0-auc:0.87732	validation_1-auc:0.81734
[42]	validation_0-auc:0.87753	validation_1-auc:0.81704
[43]	validation_0-auc:0.87762	validation_1-auc:0.81661
[44]	validation_0-auc:0.87802	validation_1-auc:0.81644
[45]	validation_0-auc:0.87904	validation_1-auc:0.81743
[46]	validation_0-auc:0.88011	validation_1-auc:0.81691
[47]	validation_0-auc:0.88005	validation_1-auc:0.81691
[48]	validation_0-auc:0.88047	validation_1-auc:0.81687
[0]	validation_0-auc:0.71266	validation_1-auc:0.69182
[1]	validation_0-auc:0.73357	validation_1-auc:0.70733
[2]	validation_0-auc:0.81857	validation_1-auc:0.78867
[3]	validation_0-auc:0.83296	validation_1-auc:0.80392
[4]	validation_0-auc:0.84052	validation_1-auc:0.80950
[5]	validation_0-auc:0.84455	validation_1-auc:0.81191
[6]	validation_0-auc:0.84420	validation_1-auc:0.80947
[7]	validation_0-auc:0.84948	validation_1-auc:0.81583
[8]	validation_0-auc:0.84734	validation_1-auc:0.81271
[9]	validation_0-auc:0.84592	validation_1-auc:0.81031
[10]	validation_0-auc:0.85141	validation_1-auc:0.81503
[11]	validation_0-auc:0.85443	validation_1-auc:0.81781
[12]	validation_0-auc:0.85613	validation_1-auc:0.81798
[13]	validation_0-auc:0.85766	validation_1-auc:0.81726
[14]	validation_0-auc:0.85860	validation_1-auc:0.81788
[15]	validation_0-auc:0.86080	validation_1-auc:0.82030
[16]	validation_0-auc:0.86127	validation_1-auc:0.82080
[17]	validation_0-auc:0.86241	validation_1-auc:0.82172
[18]	validation_0-auc:0.86401	validation_1-auc:0.82315
[19]	validation_0-auc:0.86456	validation_1-auc:0.82270
[20]	validation_0-auc:0.86500	validation_1-auc:0.82280
[21]	validation_0-auc:0.86623	validation_1-auc:0.82369
[22]	validation_0-auc:0.86663	validation_1-auc:0.82406
[23]	validation_0-auc:0.86691	validation_1-auc:0.82447
[24]	validation_0-auc:0.86777	validation_1-auc:0.82434
[25]	validation_0-auc:0.86857	validation_1-auc:0.82491
[26]	validation_0-auc:0.86986	validation_1-auc:0.82529
[27]	validation_0-auc:0.87081	validation_1-auc:0.82505
[28]	validation_0-auc:0.87114	validation_1-auc:0.82504
[29]	validation_0-auc:0.87185	validation_1-auc:0.82451
[30]	validation_0-auc:0.87239	validation_1-auc:0.82491
[31]	validation_0-auc:0.87264	validation_1-auc:0.82427
[32]	validation_0-auc:0.87305	validation_1-auc:0.82447
[33]	validation_0-auc:0.87333	validation_1-auc:0.82392
[34]	validation_0-auc:0.87376	validation_1-auc:0.82341
[35]	validation_0-auc:0.87435	validation_1-auc:0.82321
[36]	validation_0-auc:0.87527	validation_1-auc:0.82271
[37]	validation_0-auc:0.87546	validation_1-auc:0.82256
[38]	validation_0-auc:0.87577	validation_1-auc:0.82255
[39]	validation_0-auc:0.87617	validation_1-auc:0.82215
[40]	validation_0-auc:0.87628	validation_1-auc:0.82129
[41]	validation_0-auc:0.87682	validation_1-auc:0.82205
[42]	validation_0-auc:0.87756	validation_1-auc:0.82111
[43]	validation_0-auc:0.87834	validation_1-auc:0.82110
[44]	validation_0-auc:0.87842	validation_1-auc:0.82087
[45]	validation_0-auc:0.87866	validation_1-auc:0.82071
[46]	validation_0-auc:0.87880	validation_1-auc:0.82047
[47]	validation_0-auc:0.87906	validation_1-auc:0.82047
[48]	validation_0-auc:0.87927	validation_1-auc:0.82037
[49]	validation_0-auc:0.87946	validation_1-auc:0.82040
[50]	validation_0-auc:0.87953	validation_1-auc:0.82069
[51]	validation_0-auc:0.87976	validation_1-auc:0.82049
[52]	validation_0-auc:0.88000	validation_1-auc:0.81994
[53]	validation_0-auc:0.88009	validation_1-auc:0.82018
[54]	validation_0-auc:0.88084	validation_1-auc:0.81967
[55]	validation_0-auc:0.88104	validation_1-auc:0.81866
[0]	validation_0-auc:0.72758	validation_1-auc:0.70261
[1]	validation_0-auc:0.74613	validation_1-auc:0.71864
[2]	validation_0-auc:0.81886	validation_1-auc:0.78905
[3]	validation_0-auc:0.83179	validation_1-auc:0.80194
[4]	validation_0-auc:0.83920	validation_1-auc:0.80951
[5]	validation_0-auc:0.84350	validation_1-auc:0.81128
[6]	validation_0-auc:0.84127	validation_1-auc:0.80598
[7]	validation_0-auc:0.84901	validation_1-auc:0.81587
[8]	validation_0-auc:0.84697	validation_1-auc:0.81146
[9]	validation_0-auc:0.84682	validation_1-auc:0.80782
[10]	validation_0-auc:0.85246	validation_1-auc:0.81172
[11]	validation_0-auc:0.85510	validation_1-auc:0.81490
[12]	validation_0-auc:0.85754	validation_1-auc:0.81612
[13]	validation_0-auc:0.85818	validation_1-auc:0.81552
[14]	validation_0-auc:0.85853	validation_1-auc:0.81382
[15]	validation_0-auc:0.86071	validation_1-auc:0.81716
[16]	validation_0-auc:0.86155	validation_1-auc:0.81671
[17]	validation_0-auc:0.86288	validation_1-auc:0.81706
[18]	validation_0-auc:0.86444	validation_1-auc:0.81838
[19]	validation_0-auc:0.86502	validation_1-auc:0.81809
[20]	validation_0-auc:0.86602	validation_1-auc:0.81762
[21]	validation_0-auc:0.86721	validation_1-auc:0.81752
[22]	validation_0-auc:0.86764	validation_1-auc:0.81706
[23]	validation_0-auc:0.86780	validation_1-auc:0.81675
[24]	validation_0-auc:0.86837	validation_1-auc:0.81740
[25]	validation_0-auc:0.86963	validation_1-auc:0.81742
[26]	validation_0-auc:0.87025	validation_1-auc:0.81762
[27]	validation_0-auc:0.87129	validation_1-auc:0.81682
[28]	validation_0-auc:0.87158	validation_1-auc:0.81712
[29]	validation_0-auc:0.87213	validation_1-auc:0.81746
[30]	validation_0-auc:0.87258	validation_1-auc:0.81713
[31]	validation_0-auc:0.87278	validation_1-auc:0.81675
[32]	validation_0-auc:0.87404	validation_1-auc:0.81792
[33]	validation_0-auc:0.87421	validation_1-auc:0.81818
[34]	validation_0-auc:0.87462	validation_1-auc:0.81848
[35]	validation_0-auc:0.87487	validation_1-auc:0.81823
[36]	validation_0-auc:0.87524	validation_1-auc:0.81857
[37]	validation_0-auc:0.87542	validation_1-auc:0.81849
[38]	validation_0-auc:0.87556	validation_1-auc:0.81806
[39]	validation_0-auc:0.87567	validation_1-auc:0.81809
[40]	validation_0-auc:0.87601	validation_1-auc:0.81764
[41]	validation_0-auc:0.87644	validation_1-auc:0.81825
[42]	validation_0-auc:0.87703	validation_1-auc:0.81718
[43]	validation_0-auc:0.87793	validation_1-auc:0.81726
[44]	validation_0-auc:0.87808	validation_1-auc:0.81741
[45]	validation_0-auc:0.87811	validation_1-auc:0.81758
[46]	validation_0-auc:0.87926	validation_1-auc:0.81706
[47]	validation_0-auc:0.87966	validation_1-auc:0.81689
[48]	validation_0-auc:0.88049	validation_1-auc:0.81721
[49]	validation_0-auc:0.88099	validation_1-auc:0.81659
[50]	validation_0-auc:0.88116	validation_1-auc:0.81606
[51]	validation_0-auc:0.88138	validation_1-auc:0.81491
[52]	validation_0-auc:0.88141	validation_1-auc:0.81504
[53]	validation_0-auc:0.88162	validation_1-auc:0.81483
[54]	validation_0-auc:0.88172	validation_1-auc:0.81466
[55]	validation_0-auc:0.88204	validation_1-auc:0.81443
[56]	validation_0-auc:0.88215	validation_1-auc:0.81429
[57]	validation_0-auc:0.88233	validation_1-auc:0.81431
[58]	validation_0-auc:0.88243	validation_1-auc:0.81411
[59]	validation_0-auc:0.88248	validation_1-auc:0.81414
[60]	validation_0-auc:0.88256	validation_1-auc:0.81390
[61]	validation_0-auc:0.88275	validation_1-auc:0.81357
[62]	validation_0-auc:0.88293	validation_1-auc:0.81339
[63]	validation_0-auc:0.88307	validation_1-auc:0.81347
[64]	validation_0-auc:0.88363	validation_1-auc:0.81364
[65]	validation_0-auc:0.88369	validation_1-auc:0.81341
[0]	validation_0-auc:0.72734	validation_1-auc:0.70571
[1]	validation_0-auc:0.75633	validation_1-auc:0.73092
[2]	validation_0-auc:0.82422	validation_1-auc:0.79729
[3]	validation_0-auc:0.83321	validation_1-auc:0.80463
[4]	validation_0-auc:0.84118	validation_1-auc:0.81424
[5]	validation_0-auc:0.84287	validation_1-auc:0.81626
[6]	validation_0-auc:0.84162	validation_1-auc:0.81367
[7]	validation_0-auc:0.84801	validation_1-auc:0.81616
[8]	validation_0-auc:0.84728	validation_1-auc:0.81330
[9]	validation_0-auc:0.84455	validation_1-auc:0.80935
[10]	validation_0-auc:0.84984	validation_1-auc:0.81477
[11]	validation_0-auc:0.85395	validation_1-auc:0.81743
[12]	validation_0-auc:0.85668	validation_1-auc:0.82046
[13]	validation_0-auc:0.85721	validation_1-auc:0.81994
[14]	validation_0-auc:0.85753	validation_1-auc:0.81758
[15]	validation_0-auc:0.86001	validation_1-auc:0.82042
[16]	validation_0-auc:0.86077	validation_1-auc:0.81869
[17]	validation_0-auc:0.86238	validation_1-auc:0.81993
[18]	validation_0-auc:0.86378	validation_1-auc:0.82165
[19]	validation_0-auc:0.86417	validation_1-auc:0.82145
[20]	validation_0-auc:0.86501	validation_1-auc:0.82195
[21]	validation_0-auc:0.86580	validation_1-auc:0.82219
[22]	validation_0-auc:0.86618	validation_1-auc:0.82218
[23]	validation_0-auc:0.86660	validation_1-auc:0.82181
[24]	validation_0-auc:0.86718	validation_1-auc:0.82148
[25]	validation_0-auc:0.86832	validation_1-auc:0.82154
[26]	validation_0-auc:0.86929	validation_1-auc:0.82155
[27]	validation_0-auc:0.87051	validation_1-auc:0.82100
[28]	validation_0-auc:0.87130	validation_1-auc:0.82096
[29]	validation_0-auc:0.87177	validation_1-auc:0.82040
[30]	validation_0-auc:0.87195	validation_1-auc:0.82006
[31]	validation_0-auc:0.87214	validation_1-auc:0.81967
[32]	validation_0-auc:0.87240	validation_1-auc:0.81981
[33]	validation_0-auc:0.87328	validation_1-auc:0.81859
[34]	validation_0-auc:0.87360	validation_1-auc:0.81843
[35]	validation_0-auc:0.87392	validation_1-auc:0.81796
[36]	validation_0-auc:0.87454	validation_1-auc:0.81924
[37]	validation_0-auc:0.87465	validation_1-auc:0.81929
[38]	validation_0-auc:0.87555	validation_1-auc:0.81885
[39]	validation_0-auc:0.87652	validation_1-auc:0.81810
[40]	validation_0-auc:0.87708	validation_1-auc:0.81763
[41]	validation_0-auc:0.87708	validation_1-auc:0.81780
[42]	validation_0-auc:0.87741	validation_1-auc:0.81729
[43]	validation_0-auc:0.87769	validation_1-auc:0.81656
[44]	validation_0-auc:0.87801	validation_1-auc:0.81637
[45]	validation_0-auc:0.87820	validation_1-auc:0.81655
[46]	validation_0-auc:0.87859	validation_1-auc:0.81565
[47]	validation_0-auc:0.87906	validation_1-auc:0.81506
[48]	validation_0-auc:0.87912	validation_1-auc:0.81484
[49]	validation_0-auc:0.87930	validation_1-auc:0.81482
[50]	validation_0-auc:0.87982	validation_1-auc:0.81498
[51]	validation_0-auc:0.87995	validation_1-auc:0.81523
[0]	validation_0-auc:0.71678	validation_1-auc:0.69339
[1]	validation_0-auc:0.73732	validation_1-auc:0.70803
[2]	validation_0-auc:0.82644	validation_1-auc:0.78997
[3]	validation_0-auc:0.84287	validation_1-auc:0.80244
[4]	validation_0-auc:0.84926	validation_1-auc:0.80762
[5]	validation_0-auc:0.85447	validation_1-auc:0.81232
[6]	validation_0-auc:0.85560	validation_1-auc:0.80924
[7]	validation_0-auc:0.86203	validation_1-auc:0.81373
[8]	validation_0-auc:0.86224	validation_1-auc:0.81079
[9]	validation_0-auc:0.86093	validation_1-auc:0.80817
[10]	validation_0-auc:0.86841	validation_1-auc:0.81251
[11]	validation_0-auc:0.87253	validation_1-auc:0.81529
[12]	validation_0-auc:0.87562	validation_1-auc:0.81601
[13]	validation_0-auc:0.87672	validation_1-auc:0.81547
[14]	validation_0-auc:0.87793	validation_1-auc:0.81538
[15]	validation_0-auc:0.88050	validation_1-auc:0.81756
[16]	validation_0-auc:0.88090	validation_1-auc:0.81688
[17]	validation_0-auc:0.88281	validation_1-auc:0.81783
[18]	validation_0-auc:0.88489	validation_1-auc:0.81763
[19]	validation_0-auc:0.88525	validation_1-auc:0.81758
[20]	validation_0-auc:0.88658	validation_1-auc:0.81717
[21]	validation_0-auc:0.88903	validation_1-auc:0.81692
[22]	validation_0-auc:0.88955	validation_1-auc:0.81716
[23]	validation_0-auc:0.88982	validation_1-auc:0.81705
[24]	validation_0-auc:0.89031	validation_1-auc:0.81688
[25]	validation_0-auc:0.89214	validation_1-auc:0.81603
[26]	validation_0-auc:0.89343	validation_1-auc:0.81645
[27]	validation_0-auc:0.89390	validation_1-auc:0.81585
[28]	validation_0-auc:0.89433	validation_1-auc:0.81634
[29]	validation_0-auc:0.89566	validation_1-auc:0.81573
[30]	validation_0-auc:0.89586	validation_1-auc:0.81527
[31]	validation_0-auc:0.89599	validation_1-auc:0.81461
[32]	validation_0-auc:0.89657	validation_1-auc:0.81475
[33]	validation_0-auc:0.89693	validation_1-auc:0.81436
[34]	validation_0-auc:0.89722	validation_1-auc:0.81416
[35]	validation_0-auc:0.89820	validation_1-auc:0.81439
[36]	validation_0-auc:0.89853	validation_1-auc:0.81437
[37]	validation_0-auc:0.89871	validation_1-auc:0.81372
[38]	validation_0-auc:0.89907	validation_1-auc:0.81284
[39]	validation_0-auc:0.89932	validation_1-auc:0.81240
[40]	validation_0-auc:0.89938	validation_1-auc:0.81222
[41]	validation_0-auc:0.89954	validation_1-auc:0.81213
[42]	validation_0-auc:0.89959	validation_1-auc:0.81188
[43]	validation_0-auc:0.89968	validation_1-auc:0.81127
[44]	validation_0-auc:0.89973	validation_1-auc:0.81100
[45]	validation_0-auc:0.90116	validation_1-auc:0.81039
[46]	validation_0-auc:0.90174	validation_1-auc:0.80946
[0]	validation_0-auc:0.72940	validation_1-auc:0.70396
[1]	validation_0-auc:0.75027	validation_1-auc:0.72265
[2]	validation_0-auc:0.82655	validation_1-auc:0.79431
[3]	validation_0-auc:0.83968	validation_1-auc:0.80845
[4]	validation_0-auc:0.84877	validation_1-auc:0.81318
[5]	validation_0-auc:0.85393	validation_1-auc:0.81536
[6]	validation_0-auc:0.85377	validation_1-auc:0.81093
[7]	validation_0-auc:0.86072	validation_1-auc:0.81491
[8]	validation_0-auc:0.86056	validation_1-auc:0.81021
[9]	validation_0-auc:0.86138	validation_1-auc:0.80593
[10]	validation_0-auc:0.86573	validation_1-auc:0.80972
[11]	validation_0-auc:0.86963	validation_1-auc:0.81410
[12]	validation_0-auc:0.87298	validation_1-auc:0.81772
[13]	validation_0-auc:0.87526	validation_1-auc:0.81646
[14]	validation_0-auc:0.87585	validation_1-auc:0.81235
[15]	validation_0-auc:0.87904	validation_1-auc:0.81497
[16]	validation_0-auc:0.88002	validation_1-auc:0.81441
[17]	validation_0-auc:0.88191	validation_1-auc:0.81714
[18]	validation_0-auc:0.88480	validation_1-auc:0.81737
[19]	validation_0-auc:0.88575	validation_1-auc:0.81704
[20]	validation_0-auc:0.88639	validation_1-auc:0.81703
[21]	validation_0-auc:0.88760	validation_1-auc:0.81760
[22]	validation_0-auc:0.88788	validation_1-auc:0.81724
[23]	validation_0-auc:0.88803	validation_1-auc:0.81743
[24]	validation_0-auc:0.88843	validation_1-auc:0.81707
[25]	validation_0-auc:0.88984	validation_1-auc:0.81734
[26]	validation_0-auc:0.89101	validation_1-auc:0.81874
[27]	validation_0-auc:0.89206	validation_1-auc:0.81801
[28]	validation_0-auc:0.89262	validation_1-auc:0.81829
[29]	validation_0-auc:0.89333	validation_1-auc:0.81922
[30]	validation_0-auc:0.89421	validation_1-auc:0.81937
[31]	validation_0-auc:0.89484	validation_1-auc:0.81936
[32]	validation_0-auc:0.89605	validation_1-auc:0.81953
[33]	validation_0-auc:0.89633	validation_1-auc:0.81965
[34]	validation_0-auc:0.89628	validation_1-auc:0.81970
[35]	validation_0-auc:0.89679	validation_1-auc:0.81889
[36]	validation_0-auc:0.89694	validation_1-auc:0.81910
[37]	validation_0-auc:0.89777	validation_1-auc:0.81902
[38]	validation_0-auc:0.89778	validation_1-auc:0.81886
[39]	validation_0-auc:0.89788	validation_1-auc:0.81868
[40]	validation_0-auc:0.89811	validation_1-auc:0.81874
[41]	validation_0-auc:0.89820	validation_1-auc:0.81850
[42]	validation_0-auc:0.89839	validation_1-auc:0.81820
[43]	validation_0-auc:0.89840	validation_1-auc:0.81739
[44]	validation_0-auc:0.89859	validation_1-auc:0.81729
[45]	validation_0-auc:0.89864	validation_1-auc:0.81703
[46]	validation_0-auc:0.89878	validation_1-auc:0.81636
[47]	validation_0-auc:0.89883	validation_1-auc:0.81619
[48]	validation_0-auc:0.89892	validation_1-auc:0.81532
[49]	validation_0-auc:0.89922	validation_1-auc:0.81555
[50]	validation_0-auc:0.89926	validation_1-auc:0.81570
[51]	validation_0-auc:0.89920	validation_1-auc:0.81541
[52]	validation_0-auc:0.89928	validation_1-auc:0.81540
[53]	validation_0-auc:0.89913	validation_1-auc:0.81547
[54]	validation_0-auc:0.89916	validation_1-auc:0.81526
[55]	validation_0-auc:0.89925	validation_1-auc:0.81514
[56]	validation_0-auc:0.89924	validation_1-auc:0.81511
[57]	validation_0-auc:0.89978	validation_1-auc:0.81485
[58]	validation_0-auc:0.90042	validation_1-auc:0.81407
[59]	validation_0-auc:0.90045	validation_1-auc:0.81400
[60]	validation_0-auc:0.90050	validation_1-auc:0.81369
[61]	validation_0-auc:0.90133	validation_1-auc:0.81327
[62]	validation_0-auc:0.90263	validation_1-auc:0.81271
[63]	validation_0-auc:0.90311	validation_1-auc:0.81259
[64]	validation_0-auc:0.90324	validation_1-auc:0.81172
[0]	validation_0-auc:0.72905	validation_1-auc:0.70475
[1]	validation_0-auc:0.76449	validation_1-auc:0.73252
[2]	validation_0-auc:0.82992	validation_1-auc:0.79810
[3]	validation_0-auc:0.84237	validation_1-auc:0.80920
[4]	validation_0-auc:0.85108	validation_1-auc:0.81141
[5]	validation_0-auc:0.85437	validation_1-auc:0.81436
[6]	validation_0-auc:0.85394	validation_1-auc:0.81119
[7]	validation_0-auc:0.86106	validation_1-auc:0.81623
[8]	validation_0-auc:0.86080	validation_1-auc:0.81368
[9]	validation_0-auc:0.85932	validation_1-auc:0.81071
[10]	validation_0-auc:0.86595	validation_1-auc:0.81615
[11]	validation_0-auc:0.86978	validation_1-auc:0.81869
[12]	validation_0-auc:0.87269	validation_1-auc:0.81962
[13]	validation_0-auc:0.87414	validation_1-auc:0.81766
[14]	validation_0-auc:0.87534	validation_1-auc:0.81847
[15]	validation_0-auc:0.87881	validation_1-auc:0.82136
[16]	validation_0-auc:0.87944	validation_1-auc:0.82066
[17]	validation_0-auc:0.88177	validation_1-auc:0.82117
[18]	validation_0-auc:0.88402	validation_1-auc:0.82158
[19]	validation_0-auc:0.88499	validation_1-auc:0.82173
[20]	validation_0-auc:0.88557	validation_1-auc:0.82208
[21]	validation_0-auc:0.88722	validation_1-auc:0.82293
[22]	validation_0-auc:0.88790	validation_1-auc:0.82230
[23]	validation_0-auc:0.88814	validation_1-auc:0.82220
[24]	validation_0-auc:0.88835	validation_1-auc:0.82232
[25]	validation_0-auc:0.88924	validation_1-auc:0.82136
[26]	validation_0-auc:0.89071	validation_1-auc:0.82183
[27]	validation_0-auc:0.89158	validation_1-auc:0.82164
[28]	validation_0-auc:0.89214	validation_1-auc:0.82107
[29]	validation_0-auc:0.89300	validation_1-auc:0.82090
[30]	validation_0-auc:0.89317	validation_1-auc:0.82058
[31]	validation_0-auc:0.89343	validation_1-auc:0.82028
[32]	validation_0-auc:0.89392	validation_1-auc:0.82038
[33]	validation_0-auc:0.89517	validation_1-auc:0.81977
[34]	validation_0-auc:0.89521	validation_1-auc:0.81945
[35]	validation_0-auc:0.89523	validation_1-auc:0.81918
[36]	validation_0-auc:0.89538	validation_1-auc:0.81921
[37]	validation_0-auc:0.89613	validation_1-auc:0.81817
[38]	validation_0-auc:0.89691	validation_1-auc:0.81713
[39]	validation_0-auc:0.89722	validation_1-auc:0.81667
[40]	validation_0-auc:0.89742	validation_1-auc:0.81635
[41]	validation_0-auc:0.89823	validation_1-auc:0.81507
[42]	validation_0-auc:0.89837	validation_1-auc:0.81497
[43]	validation_0-auc:0.89851	validation_1-auc:0.81494
[44]	validation_0-auc:0.89859	validation_1-auc:0.81479
[45]	validation_0-auc:0.89854	validation_1-auc:0.81442
[46]	validation_0-auc:0.89849	validation_1-auc:0.81372
[47]	validation_0-auc:0.89872	validation_1-auc:0.81403
[48]	validation_0-auc:0.89913	validation_1-auc:0.81396
[49]	validation_0-auc:0.89998	validation_1-auc:0.81383
[50]	validation_0-auc:0.89992	validation_1-auc:0.81265
[51]	validation_0-auc:0.90010	validation_1-auc:0.81171
[0]	validation_0-auc:0.71575	validation_1-auc:0.69359
[1]	validation_0-auc:0.74203	validation_1-auc:0.71466
[2]	validation_0-auc:0.82691	validation_1-auc:0.79313
[3]	validation_0-auc:0.83957	validation_1-auc:0.80175
[4]	validation_0-auc:0.84848	validation_1-auc:0.80786
[5]	validation_0-auc:0.85305	validation_1-auc:0.81056
[6]	validation_0-auc:0.85249	validation_1-auc:0.80671
[7]	validation_0-auc:0.85932	validation_1-auc:0.81154
[8]	validation_0-auc:0.85911	validation_1-auc:0.80755
[9]	validation_0-auc:0.85846	validation_1-auc:0.80321
[10]	validation_0-auc:0.86499	validation_1-auc:0.80828
[11]	validation_0-auc:0.86788	validation_1-auc:0.81111
[12]	validation_0-auc:0.87070	validation_1-auc:0.81117
[13]	validation_0-auc:0.87184	validation_1-auc:0.80943
[14]	validation_0-auc:0.87257	validation_1-auc:0.81030
[15]	validation_0-auc:0.87525	validation_1-auc:0.81243
[16]	validation_0-auc:0.87614	validation_1-auc:0.81140
[17]	validation_0-auc:0.87781	validation_1-auc:0.81283
[18]	validation_0-auc:0.88036	validation_1-auc:0.81524
[19]	validation_0-auc:0.88055	validation_1-auc:0.81457
[20]	validation_0-auc:0.88120	validation_1-auc:0.81457
[21]	validation_0-auc:0.88290	validation_1-auc:0.81578
[22]	validation_0-auc:0.88331	validation_1-auc:0.81573
[23]	validation_0-auc:0.88363	validation_1-auc:0.81518
[24]	validation_0-auc:0.88407	validation_1-auc:0.81499
[25]	validation_0-auc:0.88437	validation_1-auc:0.81469
[26]	validation_0-auc:0.88564	validation_1-auc:0.81500
[27]	validation_0-auc:0.88737	validation_1-auc:0.81240
[28]	validation_0-auc:0.88758	validation_1-auc:0.81251
[29]	validation_0-auc:0.88872	validation_1-auc:0.81319
[30]	validation_0-auc:0.88900	validation_1-auc:0.81274
[31]	validation_0-auc:0.88914	validation_1-auc:0.81254
[32]	validation_0-auc:0.88958	validation_1-auc:0.81263
[33]	validation_0-auc:0.89002	validation_1-auc:0.81145
[34]	validation_0-auc:0.89019	validation_1-auc:0.81070
[35]	validation_0-auc:0.89077	validation_1-auc:0.81054
[36]	validation_0-auc:0.89096	validation_1-auc:0.81093
[37]	validation_0-auc:0.89114	validation_1-auc:0.81046
[38]	validation_0-auc:0.89139	validation_1-auc:0.81051
[39]	validation_0-auc:0.89142	validation_1-auc:0.80964
[40]	validation_0-auc:0.89216	validation_1-auc:0.80916
[41]	validation_0-auc:0.89251	validation_1-auc:0.80876
[42]	validation_0-auc:0.89268	validation_1-auc:0.80866
[43]	validation_0-auc:0.89292	validation_1-auc:0.80802
[44]	validation_0-auc:0.89378	validation_1-auc:0.80663
[45]	validation_0-auc:0.89464	validation_1-auc:0.80659
[46]	validation_0-auc:0.89466	validation_1-auc:0.80634
[47]	validation_0-auc:0.89496	validation_1-auc:0.80628
[48]	validation_0-auc:0.89553	validation_1-auc:0.80476
[49]	validation_0-auc:0.89578	validation_1-auc:0.80483
[50]	validation_0-auc:0.89649	validation_1-auc:0.80537
[51]	validation_0-auc:0.89677	validation_1-auc:0.80480
[0]	validation_0-auc:0.73196	validation_1-auc:0.70233
[1]	validation_0-auc:0.75303	validation_1-auc:0.71741
[2]	validation_0-auc:0.82804	validation_1-auc:0.79231
[3]	validation_0-auc:0.84125	validation_1-auc:0.80372
[4]	validation_0-auc:0.84782	validation_1-auc:0.80963
[5]	validation_0-auc:0.85254	validation_1-auc:0.81207
[6]	validation_0-auc:0.85156	validation_1-auc:0.80820
[7]	validation_0-auc:0.85851	validation_1-auc:0.81412
[8]	validation_0-auc:0.85884	validation_1-auc:0.80780
[9]	validation_0-auc:0.85873	validation_1-auc:0.80700
[10]	validation_0-auc:0.86476	validation_1-auc:0.81126
[11]	validation_0-auc:0.86753	validation_1-auc:0.81341
[12]	validation_0-auc:0.86940	validation_1-auc:0.81831
[13]	validation_0-auc:0.86972	validation_1-auc:0.81747
[14]	validation_0-auc:0.87100	validation_1-auc:0.81506
[15]	validation_0-auc:0.87408	validation_1-auc:0.81768
[16]	validation_0-auc:0.87555	validation_1-auc:0.81594
[17]	validation_0-auc:0.87701	validation_1-auc:0.81804
[18]	validation_0-auc:0.87956	validation_1-auc:0.81738
[19]	validation_0-auc:0.88044	validation_1-auc:0.81610
[20]	validation_0-auc:0.88104	validation_1-auc:0.81651
[21]	validation_0-auc:0.88274	validation_1-auc:0.81688
[22]	validation_0-auc:0.88300	validation_1-auc:0.81638
[23]	validation_0-auc:0.88305	validation_1-auc:0.81569
[24]	validation_0-auc:0.88342	validation_1-auc:0.81556
[25]	validation_0-auc:0.88471	validation_1-auc:0.81541
[26]	validation_0-auc:0.88540	validation_1-auc:0.81568
[27]	validation_0-auc:0.88630	validation_1-auc:0.81543
[28]	validation_0-auc:0.88708	validation_1-auc:0.81524
[29]	validation_0-auc:0.88842	validation_1-auc:0.81680
[30]	validation_0-auc:0.88864	validation_1-auc:0.81731
[31]	validation_0-auc:0.88876	validation_1-auc:0.81678
[32]	validation_0-auc:0.88951	validation_1-auc:0.81736
[33]	validation_0-auc:0.89004	validation_1-auc:0.81677
[34]	validation_0-auc:0.89158	validation_1-auc:0.81538
[35]	validation_0-auc:0.89202	validation_1-auc:0.81464
[36]	validation_0-auc:0.89253	validation_1-auc:0.81495
[37]	validation_0-auc:0.89328	validation_1-auc:0.81447
[38]	validation_0-auc:0.89393	validation_1-auc:0.81313
[39]	validation_0-auc:0.89403	validation_1-auc:0.81232
[40]	validation_0-auc:0.89410	validation_1-auc:0.81230
[41]	validation_0-auc:0.89414	validation_1-auc:0.81184
[42]	validation_0-auc:0.89438	validation_1-auc:0.81134
[0]	validation_0-auc:0.73116	validation_1-auc:0.70356
[1]	validation_0-auc:0.76390	validation_1-auc:0.73043
[2]	validation_0-auc:0.82942	validation_1-auc:0.79742
[3]	validation_0-auc:0.84110	validation_1-auc:0.81063
[4]	validation_0-auc:0.84923	validation_1-auc:0.81260
[5]	validation_0-auc:0.85193	validation_1-auc:0.81634
[6]	validation_0-auc:0.85117	validation_1-auc:0.81246
[7]	validation_0-auc:0.85843	validation_1-auc:0.81575
[8]	validation_0-auc:0.85728	validation_1-auc:0.81208
[9]	validation_0-auc:0.85568	validation_1-auc:0.80866
[10]	validation_0-auc:0.86219	validation_1-auc:0.81349
[11]	validation_0-auc:0.86581	validation_1-auc:0.81751
[12]	validation_0-auc:0.86882	validation_1-auc:0.81908
[13]	validation_0-auc:0.86954	validation_1-auc:0.81749
[14]	validation_0-auc:0.87107	validation_1-auc:0.81711
[15]	validation_0-auc:0.87388	validation_1-auc:0.81996
[16]	validation_0-auc:0.87448	validation_1-auc:0.81926
[17]	validation_0-auc:0.87592	validation_1-auc:0.82068
[18]	validation_0-auc:0.87896	validation_1-auc:0.82118
[19]	validation_0-auc:0.87982	validation_1-auc:0.82004
[20]	validation_0-auc:0.88079	validation_1-auc:0.82003
[21]	validation_0-auc:0.88196	validation_1-auc:0.81891
[22]	validation_0-auc:0.88235	validation_1-auc:0.81875
[23]	validation_0-auc:0.88261	validation_1-auc:0.81847
[24]	validation_0-auc:0.88284	validation_1-auc:0.81821
[25]	validation_0-auc:0.88385	validation_1-auc:0.81843
[26]	validation_0-auc:0.88472	validation_1-auc:0.81835
[27]	validation_0-auc:0.88518	validation_1-auc:0.81691
[28]	validation_0-auc:0.88562	validation_1-auc:0.81724
[29]	validation_0-auc:0.88625	validation_1-auc:0.81707
[30]	validation_0-auc:0.88643	validation_1-auc:0.81687
[31]	validation_0-auc:0.88669	validation_1-auc:0.81624
[32]	validation_0-auc:0.88716	validation_1-auc:0.81603
[33]	validation_0-auc:0.88759	validation_1-auc:0.81647
[34]	validation_0-auc:0.88812	validation_1-auc:0.81704
[35]	validation_0-auc:0.88836	validation_1-auc:0.81659
[36]	validation_0-auc:0.88878	validation_1-auc:0.81609
[37]	validation_0-auc:0.88887	validation_1-auc:0.81592
[38]	validation_0-auc:0.88904	validation_1-auc:0.81595
[39]	validation_0-auc:0.88938	validation_1-auc:0.81535
[40]	validation_0-auc:0.88946	validation_1-auc:0.81515
[41]	validation_0-auc:0.88966	validation_1-auc:0.81495
[42]	validation_0-auc:0.88977	validation_1-auc:0.81473
[43]	validation_0-auc:0.89098	validation_1-auc:0.81389
[44]	validation_0-auc:0.89172	validation_1-auc:0.81268
[45]	validation_0-auc:0.89189	validation_1-auc:0.81268
[46]	validation_0-auc:0.89254	validation_1-auc:0.81306
[47]	validation_0-auc:0.89259	validation_1-auc:0.81285
[48]	validation_0-auc:0.89281	validation_1-auc:0.81304
[0]	validation_0-auc:0.73009	validation_1-auc:0.69895
[1]	validation_0-auc:0.74938	validation_1-auc:0.71941
[2]	validation_0-auc:0.81594	validation_1-auc:0.78455
[3]	validation_0-auc:0.83287	validation_1-auc:0.79916
[4]	validation_0-auc:0.84058	validation_1-auc:0.80769
[5]	validation_0-auc:0.84311	validation_1-auc:0.81147
[6]	validation_0-auc:0.84343	validation_1-auc:0.80893
[7]	validation_0-auc:0.84828	validation_1-auc:0.81238
[8]	validation_0-auc:0.84793	validation_1-auc:0.81001
[9]	validation_0-auc:0.84790	validation_1-auc:0.80640
[10]	validation_0-auc:0.85357	validation_1-auc:0.81166
[11]	validation_0-auc:0.85712	validation_1-auc:0.81582
[12]	validation_0-auc:0.86052	validation_1-auc:0.81880
[13]	validation_0-auc:0.86133	validation_1-auc:0.81759
[14]	validation_0-auc:0.86310	validation_1-auc:0.81947
[15]	validation_0-auc:0.86430	validation_1-auc:0.82046
[16]	validation_0-auc:0.86611	validation_1-auc:0.82115
[17]	validation_0-auc:0.86751	validation_1-auc:0.82134
[18]	validation_0-auc:0.86814	validation_1-auc:0.82255
[19]	validation_0-auc:0.86939	validation_1-auc:0.82284
[20]	validation_0-auc:0.87067	validation_1-auc:0.82362
[21]	validation_0-auc:0.87149	validation_1-auc:0.82298
[22]	validation_0-auc:0.87245	validation_1-auc:0.82288
[23]	validation_0-auc:0.87279	validation_1-auc:0.82307
[24]	validation_0-auc:0.87358	validation_1-auc:0.82306
[25]	validation_0-auc:0.87410	validation_1-auc:0.82287
[26]	validation_0-auc:0.87523	validation_1-auc:0.82284
[27]	validation_0-auc:0.87579	validation_1-auc:0.82274
[28]	validation_0-auc:0.87609	validation_1-auc:0.82284
[29]	validation_0-auc:0.87661	validation_1-auc:0.82282
[30]	validation_0-auc:0.87694	validation_1-auc:0.82276
[31]	validation_0-auc:0.87786	validation_1-auc:0.82277
[32]	validation_0-auc:0.87822	validation_1-auc:0.82261
[33]	validation_0-auc:0.87850	validation_1-auc:0.82278
[34]	validation_0-auc:0.87879	validation_1-auc:0.82232
[35]	validation_0-auc:0.87896	validation_1-auc:0.82246
[36]	validation_0-auc:0.87915	validation_1-auc:0.82229
[37]	validation_0-auc:0.87955	validation_1-auc:0.82240
[38]	validation_0-auc:0.87973	validation_1-auc:0.82181
[39]	validation_0-auc:0.88008	validation_1-auc:0.82120
[40]	validation_0-auc:0.88029	validation_1-auc:0.82123
[41]	validation_0-auc:0.88048	validation_1-auc:0.82106
[42]	validation_0-auc:0.88081	validation_1-auc:0.82115
[43]	validation_0-auc:0.88180	validation_1-auc:0.82115
[44]	validation_0-auc:0.88245	validation_1-auc:0.82105
[45]	validation_0-auc:0.88289	validation_1-auc:0.82093
[46]	validation_0-auc:0.88321	validation_1-auc:0.82055
[47]	validation_0-auc:0.88330	validation_1-auc:0.82037
[48]	validation_0-auc:0.88348	validation_1-auc:0.82027
[49]	validation_0-auc:0.88360	validation_1-auc:0.82008
[0]	validation_0-auc:0.72992	validation_1-auc:0.70151
[1]	validation_0-auc:0.74711	validation_1-auc:0.71767
[2]	validation_0-auc:0.82110	validation_1-auc:0.78719
[3]	validation_0-auc:0.83493	validation_1-auc:0.80218
[4]	validation_0-auc:0.84194	validation_1-auc:0.81173
[5]	validation_0-auc:0.84544	validation_1-auc:0.81377
[6]	validation_0-auc:0.84438	validation_1-auc:0.81031
[7]	validation_0-auc:0.84974	validation_1-auc:0.81450
[8]	validation_0-auc:0.85072	validation_1-auc:0.81173
[9]	validation_0-auc:0.85083	validation_1-auc:0.81097
[10]	validation_0-auc:0.85501	validation_1-auc:0.81488
[11]	validation_0-auc:0.85800	validation_1-auc:0.81652
[12]	validation_0-auc:0.86066	validation_1-auc:0.81858
[13]	validation_0-auc:0.86188	validation_1-auc:0.81794
[14]	validation_0-auc:0.86374	validation_1-auc:0.81886
[15]	validation_0-auc:0.86482	validation_1-auc:0.81985
[16]	validation_0-auc:0.86613	validation_1-auc:0.82087
[17]	validation_0-auc:0.86749	validation_1-auc:0.82092
[18]	validation_0-auc:0.86849	validation_1-auc:0.82146
[19]	validation_0-auc:0.86953	validation_1-auc:0.82078
[20]	validation_0-auc:0.87004	validation_1-auc:0.82084
[21]	validation_0-auc:0.87058	validation_1-auc:0.82112
[22]	validation_0-auc:0.87122	validation_1-auc:0.82130
[23]	validation_0-auc:0.87204	validation_1-auc:0.82188
[24]	validation_0-auc:0.87317	validation_1-auc:0.82188
[25]	validation_0-auc:0.87387	validation_1-auc:0.82221
[26]	validation_0-auc:0.87497	validation_1-auc:0.82163
[27]	validation_0-auc:0.87537	validation_1-auc:0.82056
[28]	validation_0-auc:0.87565	validation_1-auc:0.81989
[29]	validation_0-auc:0.87569	validation_1-auc:0.81984
[30]	validation_0-auc:0.87618	validation_1-auc:0.81969
[31]	validation_0-auc:0.87719	validation_1-auc:0.81986
[32]	validation_0-auc:0.87773	validation_1-auc:0.81939
[33]	validation_0-auc:0.87836	validation_1-auc:0.81900
[34]	validation_0-auc:0.87843	validation_1-auc:0.81906
[35]	validation_0-auc:0.87938	validation_1-auc:0.81909
[36]	validation_0-auc:0.87954	validation_1-auc:0.81855
[37]	validation_0-auc:0.87967	validation_1-auc:0.81865
[38]	validation_0-auc:0.87984	validation_1-auc:0.81864
[39]	validation_0-auc:0.88001	validation_1-auc:0.81879
[40]	validation_0-auc:0.88037	validation_1-auc:0.81931
[41]	validation_0-auc:0.88051	validation_1-auc:0.81960
[42]	validation_0-auc:0.88084	validation_1-auc:0.81986
[43]	validation_0-auc:0.88099	validation_1-auc:0.81986
[44]	validation_0-auc:0.88125	validation_1-auc:0.81912
[45]	validation_0-auc:0.88185	validation_1-auc:0.81869
[46]	validation_0-auc:0.88192	validation_1-auc:0.81849
[47]	validation_0-auc:0.88306	validation_1-auc:0.81784
[48]	validation_0-auc:0.88316	validation_1-auc:0.81800
[49]	validation_0-auc:0.88343	validation_1-auc:0.81759
[50]	validation_0-auc:0.88350	validation_1-auc:0.81747
[51]	validation_0-auc:0.88372	validation_1-auc:0.81740
[52]	validation_0-auc:0.88386	validation_1-auc:0.81739
[53]	validation_0-auc:0.88397	validation_1-auc:0.81742
[54]	validation_0-auc:0.88473	validation_1-auc:0.81685
[55]	validation_0-auc:0.88546	validation_1-auc:0.81669
[0]	validation_0-auc:0.73701	validation_1-auc:0.70764
[1]	validation_0-auc:0.76062	validation_1-auc:0.72784
[2]	validation_0-auc:0.82070	validation_1-auc:0.78865
[3]	validation_0-auc:0.83388	validation_1-auc:0.80247
[4]	validation_0-auc:0.84206	validation_1-auc:0.81088
[5]	validation_0-auc:0.84561	validation_1-auc:0.81541
[6]	validation_0-auc:0.84328	validation_1-auc:0.80910
[7]	validation_0-auc:0.84848	validation_1-auc:0.81458
[8]	validation_0-auc:0.85012	validation_1-auc:0.81113
[9]	validation_0-auc:0.84853	validation_1-auc:0.80745
[10]	validation_0-auc:0.85406	validation_1-auc:0.81381
[11]	validation_0-auc:0.85712	validation_1-auc:0.81753
[12]	validation_0-auc:0.85924	validation_1-auc:0.82068
[13]	validation_0-auc:0.86013	validation_1-auc:0.81851
[14]	validation_0-auc:0.86256	validation_1-auc:0.82045
[15]	validation_0-auc:0.86400	validation_1-auc:0.82168
[16]	validation_0-auc:0.86538	validation_1-auc:0.82153
[17]	validation_0-auc:0.86613	validation_1-auc:0.82207
[18]	validation_0-auc:0.86750	validation_1-auc:0.82336
[19]	validation_0-auc:0.86829	validation_1-auc:0.82345
[20]	validation_0-auc:0.86979	validation_1-auc:0.82434
[21]	validation_0-auc:0.87110	validation_1-auc:0.82490
[22]	validation_0-auc:0.87171	validation_1-auc:0.82411
[23]	validation_0-auc:0.87228	validation_1-auc:0.82363
[24]	validation_0-auc:0.87338	validation_1-auc:0.82371
[25]	validation_0-auc:0.87371	validation_1-auc:0.82305
[26]	validation_0-auc:0.87373	validation_1-auc:0.82289
[27]	validation_0-auc:0.87500	validation_1-auc:0.82252
[28]	validation_0-auc:0.87554	validation_1-auc:0.82245
[29]	validation_0-auc:0.87586	validation_1-auc:0.82276
[30]	validation_0-auc:0.87590	validation_1-auc:0.82230
[31]	validation_0-auc:0.87616	validation_1-auc:0.82238
[32]	validation_0-auc:0.87723	validation_1-auc:0.82084
[33]	validation_0-auc:0.87744	validation_1-auc:0.82048
[34]	validation_0-auc:0.87772	validation_1-auc:0.82053
[35]	validation_0-auc:0.87785	validation_1-auc:0.81934
[36]	validation_0-auc:0.87795	validation_1-auc:0.81925
[37]	validation_0-auc:0.87831	validation_1-auc:0.81915
[38]	validation_0-auc:0.87937	validation_1-auc:0.81973
[39]	validation_0-auc:0.87971	validation_1-auc:0.81886
[40]	validation_0-auc:0.88016	validation_1-auc:0.81850
[41]	validation_0-auc:0.88016	validation_1-auc:0.81851
[42]	validation_0-auc:0.88075	validation_1-auc:0.81866
[43]	validation_0-auc:0.88122	validation_1-auc:0.81832
[44]	validation_0-auc:0.88148	validation_1-auc:0.81805
[45]	validation_0-auc:0.88167	validation_1-auc:0.81768
[46]	validation_0-auc:0.88189	validation_1-auc:0.81742
[47]	validation_0-auc:0.88258	validation_1-auc:0.81767
[48]	validation_0-auc:0.88259	validation_1-auc:0.81736
[49]	validation_0-auc:0.88277	validation_1-auc:0.81739
[50]	validation_0-auc:0.88287	validation_1-auc:0.81733
[51]	validation_0-auc:0.88302	validation_1-auc:0.81757
[0]	validation_0-auc:0.73477	validation_1-auc:0.69745
[1]	validation_0-auc:0.75410	validation_1-auc:0.71779
[2]	validation_0-auc:0.81646	validation_1-auc:0.78294
[3]	validation_0-auc:0.83755	validation_1-auc:0.80166
[4]	validation_0-auc:0.84253	validation_1-auc:0.80757
[5]	validation_0-auc:0.84560	validation_1-auc:0.81276
[6]	validation_0-auc:0.84564	validation_1-auc:0.81039
[7]	validation_0-auc:0.84982	validation_1-auc:0.81281
[8]	validation_0-auc:0.84836	validation_1-auc:0.81145
[9]	validation_0-auc:0.84869	validation_1-auc:0.80854
[10]	validation_0-auc:0.85401	validation_1-auc:0.81086
[11]	validation_0-auc:0.85681	validation_1-auc:0.81615
[12]	validation_0-auc:0.85983	validation_1-auc:0.81940
[13]	validation_0-auc:0.86108	validation_1-auc:0.81800
[14]	validation_0-auc:0.86251	validation_1-auc:0.82023
[15]	validation_0-auc:0.86370	validation_1-auc:0.82164
[16]	validation_0-auc:0.86426	validation_1-auc:0.82255
[17]	validation_0-auc:0.86560	validation_1-auc:0.82236
[18]	validation_0-auc:0.86616	validation_1-auc:0.82222
[19]	validation_0-auc:0.86664	validation_1-auc:0.82273
[20]	validation_0-auc:0.86738	validation_1-auc:0.82272
[21]	validation_0-auc:0.86851	validation_1-auc:0.82303
[22]	validation_0-auc:0.86922	validation_1-auc:0.82349
[23]	validation_0-auc:0.86964	validation_1-auc:0.82387
[24]	validation_0-auc:0.87020	validation_1-auc:0.82354
[25]	validation_0-auc:0.87057	validation_1-auc:0.82285
[26]	validation_0-auc:0.87105	validation_1-auc:0.82265
[27]	validation_0-auc:0.87190	validation_1-auc:0.82207
[28]	validation_0-auc:0.87235	validation_1-auc:0.82164
[29]	validation_0-auc:0.87292	validation_1-auc:0.82164
[30]	validation_0-auc:0.87370	validation_1-auc:0.82228
[31]	validation_0-auc:0.87383	validation_1-auc:0.82247
[32]	validation_0-auc:0.87403	validation_1-auc:0.82228
[33]	validation_0-auc:0.87511	validation_1-auc:0.82187
[34]	validation_0-auc:0.87598	validation_1-auc:0.82186
[35]	validation_0-auc:0.87669	validation_1-auc:0.82132
[36]	validation_0-auc:0.87688	validation_1-auc:0.82116
[37]	validation_0-auc:0.87715	validation_1-auc:0.82125
[38]	validation_0-auc:0.87769	validation_1-auc:0.82083
[39]	validation_0-auc:0.87783	validation_1-auc:0.82068
[40]	validation_0-auc:0.87809	validation_1-auc:0.82043
[41]	validation_0-auc:0.87970	validation_1-auc:0.81983
[42]	validation_0-auc:0.87991	validation_1-auc:0.81891
[43]	validation_0-auc:0.87999	validation_1-auc:0.81833
[44]	validation_0-auc:0.88017	validation_1-auc:0.81836
[45]	validation_0-auc:0.88024	validation_1-auc:0.81809
[46]	validation_0-auc:0.88027	validation_1-auc:0.81831
[47]	validation_0-auc:0.88085	validation_1-auc:0.81821
[48]	validation_0-auc:0.88097	validation_1-auc:0.81836
[49]	validation_0-auc:0.88112	validation_1-auc:0.81787
[50]	validation_0-auc:0.88128	validation_1-auc:0.81687
[51]	validation_0-auc:0.88153	validation_1-auc:0.81629
[52]	validation_0-auc:0.88168	validation_1-auc:0.81601
[53]	validation_0-auc:0.88181	validation_1-auc:0.81584
[0]	validation_0-auc:0.73183	validation_1-auc:0.70199
[1]	validation_0-auc:0.75076	validation_1-auc:0.72103
[2]	validation_0-auc:0.82202	validation_1-auc:0.78621
[3]	validation_0-auc:0.83443	validation_1-auc:0.79903
[4]	validation_0-auc:0.84121	validation_1-auc:0.80613
[5]	validation_0-auc:0.84688	validation_1-auc:0.81219
[6]	validation_0-auc:0.84418	validation_1-auc:0.80552
[7]	validation_0-auc:0.84958	validation_1-auc:0.81361
[8]	validation_0-auc:0.84998	validation_1-auc:0.81110
[9]	validation_0-auc:0.84890	validation_1-auc:0.80945
[10]	validation_0-auc:0.85402	validation_1-auc:0.81315
[11]	validation_0-auc:0.85729	validation_1-auc:0.81676
[12]	validation_0-auc:0.85943	validation_1-auc:0.81881
[13]	validation_0-auc:0.86035	validation_1-auc:0.81750
[14]	validation_0-auc:0.86178	validation_1-auc:0.81904
[15]	validation_0-auc:0.86331	validation_1-auc:0.82108
[16]	validation_0-auc:0.86459	validation_1-auc:0.82135
[17]	validation_0-auc:0.86549	validation_1-auc:0.82147
[18]	validation_0-auc:0.86628	validation_1-auc:0.82161
[19]	validation_0-auc:0.86706	validation_1-auc:0.82117
[20]	validation_0-auc:0.86805	validation_1-auc:0.82159
[21]	validation_0-auc:0.86923	validation_1-auc:0.82237
[22]	validation_0-auc:0.87005	validation_1-auc:0.82177
[23]	validation_0-auc:0.87058	validation_1-auc:0.82193
[24]	validation_0-auc:0.87090	validation_1-auc:0.82111
[25]	validation_0-auc:0.87133	validation_1-auc:0.82083
[26]	validation_0-auc:0.87265	validation_1-auc:0.82194
[27]	validation_0-auc:0.87378	validation_1-auc:0.82169
[28]	validation_0-auc:0.87423	validation_1-auc:0.82054
[29]	validation_0-auc:0.87429	validation_1-auc:0.81997
[30]	validation_0-auc:0.87445	validation_1-auc:0.81956
[31]	validation_0-auc:0.87460	validation_1-auc:0.81988
[32]	validation_0-auc:0.87549	validation_1-auc:0.81996
[33]	validation_0-auc:0.87560	validation_1-auc:0.82051
[34]	validation_0-auc:0.87616	validation_1-auc:0.82003
[35]	validation_0-auc:0.87680	validation_1-auc:0.82026
[36]	validation_0-auc:0.87698	validation_1-auc:0.82021
[37]	validation_0-auc:0.87734	validation_1-auc:0.81896
[38]	validation_0-auc:0.87766	validation_1-auc:0.81864
[39]	validation_0-auc:0.87834	validation_1-auc:0.81751
[40]	validation_0-auc:0.87849	validation_1-auc:0.81726
[41]	validation_0-auc:0.87966	validation_1-auc:0.81756
[42]	validation_0-auc:0.88009	validation_1-auc:0.81686
[43]	validation_0-auc:0.88024	validation_1-auc:0.81685
[44]	validation_0-auc:0.88061	validation_1-auc:0.81655
[45]	validation_0-auc:0.88115	validation_1-auc:0.81675
[46]	validation_0-auc:0.88173	validation_1-auc:0.81659
[47]	validation_0-auc:0.88231	validation_1-auc:0.81624
[48]	validation_0-auc:0.88272	validation_1-auc:0.81590
[49]	validation_0-auc:0.88286	validation_1-auc:0.81612
[50]	validation_0-auc:0.88289	validation_1-auc:0.81622
[51]	validation_0-auc:0.88305	validation_1-auc:0.81613
[0]	validation_0-auc:0.74115	validation_1-auc:0.70807
[1]	validation_0-auc:0.76368	validation_1-auc:0.72627
[2]	validation_0-auc:0.82339	validation_1-auc:0.79134
[3]	validation_0-auc:0.83425	validation_1-auc:0.80253
[4]	validation_0-auc:0.84102	validation_1-auc:0.80984
[5]	validation_0-auc:0.84530	validation_1-auc:0.81503
[6]	validation_0-auc:0.84292	validation_1-auc:0.80855
[7]	validation_0-auc:0.84791	validation_1-auc:0.81273
[8]	validation_0-auc:0.84938	validation_1-auc:0.80767
[9]	validation_0-auc:0.84767	validation_1-auc:0.80447
[10]	validation_0-auc:0.85278	validation_1-auc:0.81042
[11]	validation_0-auc:0.85593	validation_1-auc:0.81416
[12]	validation_0-auc:0.85874	validation_1-auc:0.81670
[13]	validation_0-auc:0.85884	validation_1-auc:0.81607
[14]	validation_0-auc:0.86056	validation_1-auc:0.81908
[15]	validation_0-auc:0.86220	validation_1-auc:0.81940
[16]	validation_0-auc:0.86416	validation_1-auc:0.82081
[17]	validation_0-auc:0.86473	validation_1-auc:0.82123
[18]	validation_0-auc:0.86610	validation_1-auc:0.82245
[19]	validation_0-auc:0.86702	validation_1-auc:0.82337
[20]	validation_0-auc:0.86801	validation_1-auc:0.82345
[21]	validation_0-auc:0.86838	validation_1-auc:0.82371
[22]	validation_0-auc:0.86988	validation_1-auc:0.82432
[23]	validation_0-auc:0.87028	validation_1-auc:0.82456
[24]	validation_0-auc:0.87063	validation_1-auc:0.82467
[25]	validation_0-auc:0.87102	validation_1-auc:0.82409
[26]	validation_0-auc:0.87126	validation_1-auc:0.82401
[27]	validation_0-auc:0.87152	validation_1-auc:0.82283
[28]	validation_0-auc:0.87186	validation_1-auc:0.82263
[29]	validation_0-auc:0.87271	validation_1-auc:0.82062
[30]	validation_0-auc:0.87317	validation_1-auc:0.81984
[31]	validation_0-auc:0.87361	validation_1-auc:0.81981
[32]	validation_0-auc:0.87389	validation_1-auc:0.81939
[33]	validation_0-auc:0.87494	validation_1-auc:0.81862
[34]	validation_0-auc:0.87507	validation_1-auc:0.81872
[35]	validation_0-auc:0.87560	validation_1-auc:0.81860
[36]	validation_0-auc:0.87632	validation_1-auc:0.81887
[37]	validation_0-auc:0.87707	validation_1-auc:0.81782
[38]	validation_0-auc:0.87716	validation_1-auc:0.81760
[39]	validation_0-auc:0.87728	validation_1-auc:0.81749
[40]	validation_0-auc:0.87803	validation_1-auc:0.81716
[41]	validation_0-auc:0.87899	validation_1-auc:0.81693
[42]	validation_0-auc:0.87917	validation_1-auc:0.81714
[43]	validation_0-auc:0.87952	validation_1-auc:0.81650
[44]	validation_0-auc:0.87967	validation_1-auc:0.81647
[45]	validation_0-auc:0.87975	validation_1-auc:0.81574
[46]	validation_0-auc:0.88004	validation_1-auc:0.81613
[47]	validation_0-auc:0.88104	validation_1-auc:0.81591
[48]	validation_0-auc:0.88153	validation_1-auc:0.81564
[49]	validation_0-auc:0.88186	validation_1-auc:0.81535
[50]	validation_0-auc:0.88201	validation_1-auc:0.81534
[51]	validation_0-auc:0.88210	validation_1-auc:0.81546
[52]	validation_0-auc:0.88240	validation_1-auc:0.81499
[53]	validation_0-auc:0.88236	validation_1-auc:0.81491
[54]	validation_0-auc:0.88259	validation_1-auc:0.81454
[0]	validation_0-auc:0.73986	validation_1-auc:0.70597
[1]	validation_0-auc:0.75945	validation_1-auc:0.72283
[2]	validation_0-auc:0.82930	validation_1-auc:0.79022
[3]	validation_0-auc:0.84410	validation_1-auc:0.80515
[4]	validation_0-auc:0.85047	validation_1-auc:0.81141
[5]	validation_0-auc:0.85570	validation_1-auc:0.81545
[6]	validation_0-auc:0.85670	validation_1-auc:0.81353
[7]	validation_0-auc:0.86354	validation_1-auc:0.81372
[8]	validation_0-auc:0.86467	validation_1-auc:0.80993
[9]	validation_0-auc:0.86440	validation_1-auc:0.80691
[10]	validation_0-auc:0.86981	validation_1-auc:0.81064
[11]	validation_0-auc:0.87419	validation_1-auc:0.81603
[12]	validation_0-auc:0.87695	validation_1-auc:0.81650
[13]	validation_0-auc:0.87824	validation_1-auc:0.81415
[14]	validation_0-auc:0.88138	validation_1-auc:0.81424
[15]	validation_0-auc:0.88273	validation_1-auc:0.81603
[16]	validation_0-auc:0.88544	validation_1-auc:0.81696
[17]	validation_0-auc:0.88732	validation_1-auc:0.81827
[18]	validation_0-auc:0.88902	validation_1-auc:0.81907
[19]	validation_0-auc:0.88992	validation_1-auc:0.81926
[20]	validation_0-auc:0.89044	validation_1-auc:0.81883
[21]	validation_0-auc:0.89198	validation_1-auc:0.81836
[22]	validation_0-auc:0.89222	validation_1-auc:0.81833
[23]	validation_0-auc:0.89248	validation_1-auc:0.81841
[24]	validation_0-auc:0.89304	validation_1-auc:0.81812
[25]	validation_0-auc:0.89341	validation_1-auc:0.81788
[26]	validation_0-auc:0.89365	validation_1-auc:0.81767
[27]	validation_0-auc:0.89421	validation_1-auc:0.81770
[28]	validation_0-auc:0.89487	validation_1-auc:0.81737
[29]	validation_0-auc:0.89609	validation_1-auc:0.81644
[30]	validation_0-auc:0.89768	validation_1-auc:0.81583
[31]	validation_0-auc:0.89809	validation_1-auc:0.81594
[32]	validation_0-auc:0.89830	validation_1-auc:0.81554
[33]	validation_0-auc:0.89922	validation_1-auc:0.81370
[34]	validation_0-auc:0.89927	validation_1-auc:0.81342
[35]	validation_0-auc:0.90077	validation_1-auc:0.81160
[36]	validation_0-auc:0.90099	validation_1-auc:0.81218
[37]	validation_0-auc:0.90115	validation_1-auc:0.81202
[38]	validation_0-auc:0.90126	validation_1-auc:0.81166
[39]	validation_0-auc:0.90140	validation_1-auc:0.81127
[40]	validation_0-auc:0.90155	validation_1-auc:0.81075
[41]	validation_0-auc:0.90151	validation_1-auc:0.81042
[42]	validation_0-auc:0.90159	validation_1-auc:0.81030
[43]	validation_0-auc:0.90194	validation_1-auc:0.80974
[44]	validation_0-auc:0.90202	validation_1-auc:0.80950
[45]	validation_0-auc:0.90208	validation_1-auc:0.80921
[46]	validation_0-auc:0.90230	validation_1-auc:0.80921
[47]	validation_0-auc:0.90264	validation_1-auc:0.80893
[48]	validation_0-auc:0.90288	validation_1-auc:0.80844
[49]	validation_0-auc:0.90350	validation_1-auc:0.80806
[0]	validation_0-auc:0.73690	validation_1-auc:0.70091
[1]	validation_0-auc:0.76426	validation_1-auc:0.72133
[2]	validation_0-auc:0.83234	validation_1-auc:0.78407
[3]	validation_0-auc:0.84550	validation_1-auc:0.79929
[4]	validation_0-auc:0.85187	validation_1-auc:0.80557
[5]	validation_0-auc:0.85747	validation_1-auc:0.81092
[6]	validation_0-auc:0.85664	validation_1-auc:0.80464
[7]	validation_0-auc:0.86359	validation_1-auc:0.81113
[8]	validation_0-auc:0.86415	validation_1-auc:0.80617
[9]	validation_0-auc:0.86482	validation_1-auc:0.80443
[10]	validation_0-auc:0.87031	validation_1-auc:0.81011
[11]	validation_0-auc:0.87421	validation_1-auc:0.81382
[12]	validation_0-auc:0.87671	validation_1-auc:0.81642
[13]	validation_0-auc:0.87769	validation_1-auc:0.81438
[14]	validation_0-auc:0.88024	validation_1-auc:0.81677
[15]	validation_0-auc:0.88214	validation_1-auc:0.81774
[16]	validation_0-auc:0.88335	validation_1-auc:0.81937
[17]	validation_0-auc:0.88457	validation_1-auc:0.81923
[18]	validation_0-auc:0.88584	validation_1-auc:0.81955
[19]	validation_0-auc:0.88680	validation_1-auc:0.81992
[20]	validation_0-auc:0.88785	validation_1-auc:0.82057
[21]	validation_0-auc:0.88855	validation_1-auc:0.81962
[22]	validation_0-auc:0.88881	validation_1-auc:0.81936
[23]	validation_0-auc:0.88971	validation_1-auc:0.81941
[24]	validation_0-auc:0.89074	validation_1-auc:0.81943
[25]	validation_0-auc:0.89109	validation_1-auc:0.81846
[26]	validation_0-auc:0.89159	validation_1-auc:0.81806
[27]	validation_0-auc:0.89209	validation_1-auc:0.81829
[28]	validation_0-auc:0.89219	validation_1-auc:0.81822
[29]	validation_0-auc:0.89251	validation_1-auc:0.81806
[30]	validation_0-auc:0.89296	validation_1-auc:0.81805
[31]	validation_0-auc:0.89300	validation_1-auc:0.81800
[32]	validation_0-auc:0.89361	validation_1-auc:0.81719
[33]	validation_0-auc:0.89369	validation_1-auc:0.81691
[34]	validation_0-auc:0.89395	validation_1-auc:0.81664
[35]	validation_0-auc:0.89412	validation_1-auc:0.81591
[36]	validation_0-auc:0.89424	validation_1-auc:0.81583
[37]	validation_0-auc:0.89469	validation_1-auc:0.81583
[38]	validation_0-auc:0.89449	validation_1-auc:0.81522
[39]	validation_0-auc:0.89450	validation_1-auc:0.81521
[40]	validation_0-auc:0.89457	validation_1-auc:0.81539
[41]	validation_0-auc:0.89460	validation_1-auc:0.81537
[42]	validation_0-auc:0.89548	validation_1-auc:0.81453
[43]	validation_0-auc:0.89560	validation_1-auc:0.81386
[44]	validation_0-auc:0.89684	validation_1-auc:0.81304
[45]	validation_0-auc:0.89770	validation_1-auc:0.81278
[46]	validation_0-auc:0.89856	validation_1-auc:0.81241
[47]	validation_0-auc:0.89910	validation_1-auc:0.81281
[48]	validation_0-auc:0.89943	validation_1-auc:0.81219
[49]	validation_0-auc:0.89952	validation_1-auc:0.81182
[50]	validation_0-auc:0.89976	validation_1-auc:0.81148
[0]	validation_0-auc:0.74553	validation_1-auc:0.70804
[1]	validation_0-auc:0.77168	validation_1-auc:0.73207
[2]	validation_0-auc:0.83109	validation_1-auc:0.79305
[3]	validation_0-auc:0.84309	validation_1-auc:0.80583
[4]	validation_0-auc:0.84930	validation_1-auc:0.81060
[5]	validation_0-auc:0.85436	validation_1-auc:0.81298
[6]	validation_0-auc:0.85268	validation_1-auc:0.80907
[7]	validation_0-auc:0.86071	validation_1-auc:0.81457
[8]	validation_0-auc:0.86444	validation_1-auc:0.80882
[9]	validation_0-auc:0.86499	validation_1-auc:0.80358
[10]	validation_0-auc:0.87216	validation_1-auc:0.80950
[11]	validation_0-auc:0.87583	validation_1-auc:0.81323
[12]	validation_0-auc:0.87913	validation_1-auc:0.81539
[13]	validation_0-auc:0.88025	validation_1-auc:0.81172
[14]	validation_0-auc:0.88292	validation_1-auc:0.81482
[15]	validation_0-auc:0.88449	validation_1-auc:0.81643
[16]	validation_0-auc:0.88548	validation_1-auc:0.81740
[17]	validation_0-auc:0.88673	validation_1-auc:0.81742
[18]	validation_0-auc:0.88859	validation_1-auc:0.81851
[19]	validation_0-auc:0.89006	validation_1-auc:0.81860
[20]	validation_0-auc:0.89045	validation_1-auc:0.81933
[21]	validation_0-auc:0.89155	validation_1-auc:0.81820
[22]	validation_0-auc:0.89171	validation_1-auc:0.81740
[23]	validation_0-auc:0.89304	validation_1-auc:0.81665
[24]	validation_0-auc:0.89425	validation_1-auc:0.81569
[25]	validation_0-auc:0.89451	validation_1-auc:0.81529
[26]	validation_0-auc:0.89568	validation_1-auc:0.81517
[27]	validation_0-auc:0.89680	validation_1-auc:0.81399
[28]	validation_0-auc:0.89736	validation_1-auc:0.81371
[29]	validation_0-auc:0.89755	validation_1-auc:0.81356
[30]	validation_0-auc:0.89789	validation_1-auc:0.81332
[31]	validation_0-auc:0.89839	validation_1-auc:0.81268
[32]	validation_0-auc:0.89912	validation_1-auc:0.81226
[33]	validation_0-auc:0.89915	validation_1-auc:0.81182
[34]	validation_0-auc:0.89945	validation_1-auc:0.81185
[35]	validation_0-auc:0.89971	validation_1-auc:0.81118
[36]	validation_0-auc:0.89981	validation_1-auc:0.81060
[37]	validation_0-auc:0.89996	validation_1-auc:0.81105
[38]	validation_0-auc:0.90019	validation_1-auc:0.81079
[39]	validation_0-auc:0.90031	validation_1-auc:0.81066
[40]	validation_0-auc:0.90035	validation_1-auc:0.81045
[41]	validation_0-auc:0.90030	validation_1-auc:0.81021
[42]	validation_0-auc:0.90036	validation_1-auc:0.81013
[43]	validation_0-auc:0.90046	validation_1-auc:0.80986
[44]	validation_0-auc:0.90053	validation_1-auc:0.80941
[45]	validation_0-auc:0.90089	validation_1-auc:0.80920
[46]	validation_0-auc:0.90094	validation_1-auc:0.80925
[47]	validation_0-auc:0.90096	validation_1-auc:0.80929
[48]	validation_0-auc:0.90168	validation_1-auc:0.80934
[49]	validation_0-auc:0.90243	validation_1-auc:0.80844
[50]	validation_0-auc:0.90247	validation_1-auc:0.80797
[0]	validation_0-auc:0.74089	validation_1-auc:0.70149
[1]	validation_0-auc:0.76083	validation_1-auc:0.71914
[2]	validation_0-auc:0.82876	validation_1-auc:0.78438
[3]	validation_0-auc:0.84505	validation_1-auc:0.80323
[4]	validation_0-auc:0.85121	validation_1-auc:0.81286
[5]	validation_0-auc:0.85639	validation_1-auc:0.81661
[6]	validation_0-auc:0.85659	validation_1-auc:0.81142
[7]	validation_0-auc:0.86077	validation_1-auc:0.81416
[8]	validation_0-auc:0.86141	validation_1-auc:0.81339
[9]	validation_0-auc:0.86082	validation_1-auc:0.80962
[10]	validation_0-auc:0.86693	validation_1-auc:0.81352
[11]	validation_0-auc:0.87070	validation_1-auc:0.81907
[12]	validation_0-auc:0.87244	validation_1-auc:0.82024
[13]	validation_0-auc:0.87341	validation_1-auc:0.81818
[14]	validation_0-auc:0.87590	validation_1-auc:0.82000
[15]	validation_0-auc:0.87793	validation_1-auc:0.82136
[16]	validation_0-auc:0.87954	validation_1-auc:0.82213
[17]	validation_0-auc:0.88123	validation_1-auc:0.82108
[18]	validation_0-auc:0.88263	validation_1-auc:0.82124
[19]	validation_0-auc:0.88340	validation_1-auc:0.82119
[20]	validation_0-auc:0.88434	validation_1-auc:0.82065
[21]	validation_0-auc:0.88628	validation_1-auc:0.81974
[22]	validation_0-auc:0.88671	validation_1-auc:0.81942
[23]	validation_0-auc:0.88705	validation_1-auc:0.81943
[24]	validation_0-auc:0.88836	validation_1-auc:0.81921
[25]	validation_0-auc:0.88869	validation_1-auc:0.81879
[26]	validation_0-auc:0.88951	validation_1-auc:0.81827
[27]	validation_0-auc:0.88988	validation_1-auc:0.81823
[28]	validation_0-auc:0.89020	validation_1-auc:0.81766
[29]	validation_0-auc:0.89040	validation_1-auc:0.81769
[30]	validation_0-auc:0.89079	validation_1-auc:0.81774
[31]	validation_0-auc:0.89131	validation_1-auc:0.81767
[32]	validation_0-auc:0.89143	validation_1-auc:0.81719
[33]	validation_0-auc:0.89264	validation_1-auc:0.81564
[34]	validation_0-auc:0.89366	validation_1-auc:0.81448
[35]	validation_0-auc:0.89447	validation_1-auc:0.81267
[36]	validation_0-auc:0.89453	validation_1-auc:0.81246
[37]	validation_0-auc:0.89463	validation_1-auc:0.81231
[38]	validation_0-auc:0.89484	validation_1-auc:0.81188
[39]	validation_0-auc:0.89504	validation_1-auc:0.81159
[40]	validation_0-auc:0.89532	validation_1-auc:0.81100
[41]	validation_0-auc:0.89586	validation_1-auc:0.81003
[42]	validation_0-auc:0.89588	validation_1-auc:0.81029
[43]	validation_0-auc:0.89630	validation_1-auc:0.81047
[44]	validation_0-auc:0.89658	validation_1-auc:0.81000
[45]	validation_0-auc:0.89741	validation_1-auc:0.80971
[46]	validation_0-auc:0.89746	validation_1-auc:0.80939
[0]	validation_0-auc:0.73590	validation_1-auc:0.70153
[1]	validation_0-auc:0.75641	validation_1-auc:0.71786
[2]	validation_0-auc:0.83166	validation_1-auc:0.79014
[3]	validation_0-auc:0.84355	validation_1-auc:0.80064
[4]	validation_0-auc:0.84944	validation_1-auc:0.80438
[5]	validation_0-auc:0.85607	validation_1-auc:0.80951
[6]	validation_0-auc:0.85630	validation_1-auc:0.80558
[7]	validation_0-auc:0.86255	validation_1-auc:0.81222
[8]	validation_0-auc:0.86191	validation_1-auc:0.80818
[9]	validation_0-auc:0.86155	validation_1-auc:0.80538
[10]	validation_0-auc:0.86788	validation_1-auc:0.80942
[11]	validation_0-auc:0.87147	validation_1-auc:0.81438
[12]	validation_0-auc:0.87449	validation_1-auc:0.81536
[13]	validation_0-auc:0.87541	validation_1-auc:0.81525
[14]	validation_0-auc:0.87697	validation_1-auc:0.81558
[15]	validation_0-auc:0.87840	validation_1-auc:0.81586
[16]	validation_0-auc:0.87991	validation_1-auc:0.81851
[17]	validation_0-auc:0.88129	validation_1-auc:0.81850
[18]	validation_0-auc:0.88161	validation_1-auc:0.81871
[19]	validation_0-auc:0.88251	validation_1-auc:0.81958
[20]	validation_0-auc:0.88325	validation_1-auc:0.81942
[21]	validation_0-auc:0.88428	validation_1-auc:0.81845
[22]	validation_0-auc:0.88468	validation_1-auc:0.81756
[23]	validation_0-auc:0.88522	validation_1-auc:0.81767
[24]	validation_0-auc:0.88536	validation_1-auc:0.81751
[25]	validation_0-auc:0.88581	validation_1-auc:0.81677
[26]	validation_0-auc:0.88603	validation_1-auc:0.81596
[27]	validation_0-auc:0.88667	validation_1-auc:0.81546
[28]	validation_0-auc:0.88700	validation_1-auc:0.81520
[29]	validation_0-auc:0.88722	validation_1-auc:0.81468
[30]	validation_0-auc:0.88730	validation_1-auc:0.81449
[31]	validation_0-auc:0.88745	validation_1-auc:0.81466
[32]	validation_0-auc:0.88769	validation_1-auc:0.81520
[33]	validation_0-auc:0.88785	validation_1-auc:0.81555
[34]	validation_0-auc:0.88843	validation_1-auc:0.81491
[35]	validation_0-auc:0.88855	validation_1-auc:0.81405
[36]	validation_0-auc:0.88902	validation_1-auc:0.81343
[37]	validation_0-auc:0.88986	validation_1-auc:0.81283
[38]	validation_0-auc:0.89087	validation_1-auc:0.81265
[39]	validation_0-auc:0.89135	validation_1-auc:0.81184
[40]	validation_0-auc:0.89174	validation_1-auc:0.81129
[41]	validation_0-auc:0.89186	validation_1-auc:0.81167
[42]	validation_0-auc:0.89215	validation_1-auc:0.81135
[43]	validation_0-auc:0.89348	validation_1-auc:0.81166
[44]	validation_0-auc:0.89481	validation_1-auc:0.81158
[45]	validation_0-auc:0.89485	validation_1-auc:0.81175
[46]	validation_0-auc:0.89556	validation_1-auc:0.81211
[47]	validation_0-auc:0.89558	validation_1-auc:0.81232
[48]	validation_0-auc:0.89575	validation_1-auc:0.81197
[49]	validation_0-auc:0.89610	validation_1-auc:0.81199
[0]	validation_0-auc:0.74669	validation_1-auc:0.70962
[1]	validation_0-auc:0.77182	validation_1-auc:0.73046
[2]	validation_0-auc:0.83057	validation_1-auc:0.79138
[3]	validation_0-auc:0.84312	validation_1-auc:0.80332
[4]	validation_0-auc:0.84876	validation_1-auc:0.80853
[5]	validation_0-auc:0.85430	validation_1-auc:0.81219
[6]	validation_0-auc:0.85286	validation_1-auc:0.80694
[7]	validation_0-auc:0.85994	validation_1-auc:0.81115
[8]	validation_0-auc:0.86251	validation_1-auc:0.80635
[9]	validation_0-auc:0.86128	validation_1-auc:0.80151
[10]	validation_0-auc:0.86766	validation_1-auc:0.80694
[11]	validation_0-auc:0.87141	validation_1-auc:0.81059
[12]	validation_0-auc:0.87453	validation_1-auc:0.81576
[13]	validation_0-auc:0.87484	validation_1-auc:0.81134
[14]	validation_0-auc:0.87724	validation_1-auc:0.81303
[15]	validation_0-auc:0.87855	validation_1-auc:0.81366
[16]	validation_0-auc:0.87971	validation_1-auc:0.81267
[17]	validation_0-auc:0.88110	validation_1-auc:0.81307
[18]	validation_0-auc:0.88247	validation_1-auc:0.81521
[19]	validation_0-auc:0.88286	validation_1-auc:0.81591
[20]	validation_0-auc:0.88435	validation_1-auc:0.81583
[21]	validation_0-auc:0.88503	validation_1-auc:0.81612
[22]	validation_0-auc:0.88531	validation_1-auc:0.81522
[23]	validation_0-auc:0.88552	validation_1-auc:0.81496
[24]	validation_0-auc:0.88629	validation_1-auc:0.81440
[25]	validation_0-auc:0.88669	validation_1-auc:0.81436
[26]	validation_0-auc:0.88709	validation_1-auc:0.81375
[27]	validation_0-auc:0.88786	validation_1-auc:0.81375
[28]	validation_0-auc:0.88807	validation_1-auc:0.81336
[29]	validation_0-auc:0.88910	validation_1-auc:0.81296
[30]	validation_0-auc:0.88957	validation_1-auc:0.81274
[31]	validation_0-auc:0.88972	validation_1-auc:0.81208
[32]	validation_0-auc:0.89040	validation_1-auc:0.81173
[33]	validation_0-auc:0.89089	validation_1-auc:0.81181
[34]	validation_0-auc:0.89108	validation_1-auc:0.81150
[35]	validation_0-auc:0.89156	validation_1-auc:0.81106
[36]	validation_0-auc:0.89181	validation_1-auc:0.81092
[37]	validation_0-auc:0.89184	validation_1-auc:0.81059
[38]	validation_0-auc:0.89201	validation_1-auc:0.81021
[39]	validation_0-auc:0.89245	validation_1-auc:0.81012
[40]	validation_0-auc:0.89262	validation_1-auc:0.80919
[41]	validation_0-auc:0.89343	validation_1-auc:0.80846
[42]	validation_0-auc:0.89366	validation_1-auc:0.80854
[43]	validation_0-auc:0.89465	validation_1-auc:0.80804
[44]	validation_0-auc:0.89493	validation_1-auc:0.80753
[45]	validation_0-auc:0.89537	validation_1-auc:0.80682
[46]	validation_0-auc:0.89547	validation_1-auc:0.80684
[47]	validation_0-auc:0.89584	validation_1-auc:0.80740
[48]	validation_0-auc:0.89598	validation_1-auc:0.80733
[49]	validation_0-auc:0.89602	validation_1-auc:0.80697
[50]	validation_0-auc:0.89611	validation_1-auc:0.80680
[0]	validation_0-auc:0.72979	validation_1-auc:0.70466
[1]	validation_0-auc:0.75464	validation_1-auc:0.72441
[2]	validation_0-auc:0.82659	validation_1-auc:0.79544
[3]	validation_0-auc:0.83885	validation_1-auc:0.80461
[4]	validation_0-auc:0.84549	validation_1-auc:0.81055
[5]	validation_0-auc:0.84873	validation_1-auc:0.81264
[6]	validation_0-auc:0.84715	validation_1-auc:0.80925
[7]	validation_0-auc:0.85207	validation_1-auc:0.81328
[8]	validation_0-auc:0.85173	validation_1-auc:0.81038
[9]	validation_0-auc:0.85052	validation_1-auc:0.80919
[10]	validation_0-auc:0.85644	validation_1-auc:0.81514
[11]	validation_0-auc:0.85887	validation_1-auc:0.81768
[12]	validation_0-auc:0.86102	validation_1-auc:0.82092
[13]	validation_0-auc:0.86264	validation_1-auc:0.82019
[14]	validation_0-auc:0.86387	validation_1-auc:0.81946
[15]	validation_0-auc:0.86590	validation_1-auc:0.82173
[16]	validation_0-auc:0.86601	validation_1-auc:0.82081
[17]	validation_0-auc:0.86770	validation_1-auc:0.82241
[18]	validation_0-auc:0.86923	validation_1-auc:0.82363
[19]	validation_0-auc:0.86999	validation_1-auc:0.82270
[20]	validation_0-auc:0.87048	validation_1-auc:0.82280
[21]	validation_0-auc:0.87215	validation_1-auc:0.82339
[22]	validation_0-auc:0.87291	validation_1-auc:0.82378
[23]	validation_0-auc:0.87370	validation_1-auc:0.82353
[24]	validation_0-auc:0.87417	validation_1-auc:0.82404
[25]	validation_0-auc:0.87572	validation_1-auc:0.82415
[26]	validation_0-auc:0.87728	validation_1-auc:0.82454
[27]	validation_0-auc:0.87906	validation_1-auc:0.82398
[28]	validation_0-auc:0.87962	validation_1-auc:0.82365
[29]	validation_0-auc:0.88044	validation_1-auc:0.82365
[30]	validation_0-auc:0.88068	validation_1-auc:0.82344
[31]	validation_0-auc:0.88101	validation_1-auc:0.82333
[32]	validation_0-auc:0.88128	validation_1-auc:0.82318
[33]	validation_0-auc:0.88219	validation_1-auc:0.82364
[34]	validation_0-auc:0.88335	validation_1-auc:0.82347
[35]	validation_0-auc:0.88384	validation_1-auc:0.82291
[36]	validation_0-auc:0.88440	validation_1-auc:0.82244
[37]	validation_0-auc:0.88518	validation_1-auc:0.82183
[38]	validation_0-auc:0.88540	validation_1-auc:0.82186
[39]	validation_0-auc:0.88590	validation_1-auc:0.82180
[40]	validation_0-auc:0.88642	validation_1-auc:0.82177
[41]	validation_0-auc:0.88680	validation_1-auc:0.82170
[42]	validation_0-auc:0.88734	validation_1-auc:0.82093
[43]	validation_0-auc:0.88752	validation_1-auc:0.82096
[44]	validation_0-auc:0.88761	validation_1-auc:0.82092
[45]	validation_0-auc:0.88778	validation_1-auc:0.82107
[46]	validation_0-auc:0.88800	validation_1-auc:0.82112
[47]	validation_0-auc:0.88913	validation_1-auc:0.82062
[48]	validation_0-auc:0.88936	validation_1-auc:0.82047
[49]	validation_0-auc:0.88943	validation_1-auc:0.82024
[50]	validation_0-auc:0.88980	validation_1-auc:0.82017
[51]	validation_0-auc:0.89026	validation_1-auc:0.82000
[52]	validation_0-auc:0.89069	validation_1-auc:0.82036
[53]	validation_0-auc:0.89092	validation_1-auc:0.82032
[54]	validation_0-auc:0.89106	validation_1-auc:0.82022
[55]	validation_0-auc:0.89192	validation_1-auc:0.82077
[56]	validation_0-auc:0.89219	validation_1-auc:0.82065
CPU times: user 39min 50s, sys: 18.1 s, total: 40min 8s
Wall time: 11min 25s

GridSearchCV(cv=3,
             estimator=XGBClassifier(base_score=None, booster=None,
                                     callbacks=None, colsample_bylevel=None,
                                     colsample_bynode=None,
                                     colsample_bytree=None,
                                     early_stopping_rounds=None,
                                     enable_categorical=False, eval_metric=None,
                                     gamma=None, gpu_id=None, grow_policy=None,
                                     importance_type=None,
                                     interaction_constraints=None,
                                     learning_rate=None, max_bin=None,
                                     max_cat_to_onehot=None,
                                     max_delta_step=None, max_depth=None,
                                     max_leaves=None, min_child_weight=None,
                                     missing=nan, monotone_constraints=None,
                                     n_estimators=100, n_jobs=None,
                                     num_parallel_tree=None, predictor=None,
                                     random_state=None, reg_alpha=None,
                                     reg_lambda=None, ...),
             param_grid={'colsample_bytree': [0.5, 0.75], 'max_depth': [5, 7],
                         'min_child_weight': [1, 3]})

print("GridSearchCV  최적 파라미터 : ", gridcv.best_params_ )

pred_prob = gridcv.predict_proba(X_test)[:, 1]

# average='macro' : 각 레이블에 대한 측정항목을 계산하고 가중치가 적용되지 않은 평균을 찾습니다.
# default=’macro’ 
xgb_roc_score = roc_auc_score(y_test, pred_prob, average='macro')
print("ROC AUC : {0:4f}".format(xgb_roc_score))

GridSearchCV  최적 파라미터 :  {'colsample_bytree': 0.5, 'max_depth': 5, 'min_child_weight': 3}
ROC AUC : 0.824543

실습해 보기

• colsample_bytree : 0.5, max_depth : 5, min_child_weight : 3로 설정
• n_estimators = 1000으로 증가, learning_rate를 조정해보고, reg_alpha를 추가하여 ROC_AUC의 값을 구해보자.

%%time

xgb_model_l = XGBClassifier(n_estimators=1000,
                            random_state= 77, 
                            learning_rate=0.02,
                            max_depth=5,
                            min_child_weight=3,
                            colsample_bytree=0.5, 
                            reg_alpha=0.03)

# 성능 평가 지표를 auc로, 조기 중단 파라미터 값은 200으로 설정하고 학습 수행
xgb_model_l.fit(X_train, y_train, 
                early_stopping_rounds=200,
                eval_metric='auc', 
                eval_set=[(X_train, y_train), (X_test, y_test)])

/Users/toto/Documents/anaconda3/lib/python3.8/site-packages/xgboost/sklearn.py:793: UserWarning: `eval_metric` in `fit` method is deprecated for better compatibility with scikit-learn, use `eval_metric` in constructor or`set_params` instead.
  warnings.warn(
/Users/toto/Documents/anaconda3/lib/python3.8/site-packages/xgboost/sklearn.py:793: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(

[0]	validation_0-auc:0.74152	validation_1-auc:0.71138
[1]	validation_0-auc:0.81652	validation_1-auc:0.78519
[2]	validation_0-auc:0.81402	validation_1-auc:0.78153
[3]	validation_0-auc:0.80746	validation_1-auc:0.77182
[4]	validation_0-auc:0.80893	validation_1-auc:0.77279
[5]	validation_0-auc:0.80719	validation_1-auc:0.77030
[6]	validation_0-auc:0.81916	validation_1-auc:0.78356
[7]	validation_0-auc:0.82535	validation_1-auc:0.79050
[8]	validation_0-auc:0.82920	validation_1-auc:0.79486
[9]	validation_0-auc:0.83281	validation_1-auc:0.79974
[10]	validation_0-auc:0.82985	validation_1-auc:0.79685
[11]	validation_0-auc:0.83332	validation_1-auc:0.80047
[12]	validation_0-auc:0.83588	validation_1-auc:0.80385
[13]	validation_0-auc:0.83746	validation_1-auc:0.80522
[14]	validation_0-auc:0.83619	validation_1-auc:0.80302
[15]	validation_0-auc:0.83675	validation_1-auc:0.80428
[16]	validation_0-auc:0.83792	validation_1-auc:0.80547
[17]	validation_0-auc:0.83954	validation_1-auc:0.80735
[18]	validation_0-auc:0.84054	validation_1-auc:0.80810
[19]	validation_0-auc:0.84129	validation_1-auc:0.80890
[20]	validation_0-auc:0.84212	validation_1-auc:0.81002
[21]	validation_0-auc:0.84300	validation_1-auc:0.81071
[22]	validation_0-auc:0.84257	validation_1-auc:0.80958
[23]	validation_0-auc:0.84358	validation_1-auc:0.81092
[24]	validation_0-auc:0.84247	validation_1-auc:0.80970
[25]	validation_0-auc:0.84290	validation_1-auc:0.81020
[26]	validation_0-auc:0.84266	validation_1-auc:0.80887
[27]	validation_0-auc:0.84345	validation_1-auc:0.80985
[28]	validation_0-auc:0.84267	validation_1-auc:0.80793
[29]	validation_0-auc:0.84334	validation_1-auc:0.80876
[30]	validation_0-auc:0.84256	validation_1-auc:0.80753
[31]	validation_0-auc:0.84307	validation_1-auc:0.80821
[32]	validation_0-auc:0.84253	validation_1-auc:0.80708
[33]	validation_0-auc:0.84316	validation_1-auc:0.80759
[34]	validation_0-auc:0.84369	validation_1-auc:0.80828
[35]	validation_0-auc:0.84350	validation_1-auc:0.80728
[36]	validation_0-auc:0.84398	validation_1-auc:0.80810
[37]	validation_0-auc:0.84461	validation_1-auc:0.80912
[38]	validation_0-auc:0.84495	validation_1-auc:0.80942
[39]	validation_0-auc:0.84538	validation_1-auc:0.80996
[40]	validation_0-auc:0.84477	validation_1-auc:0.80972
[41]	validation_0-auc:0.84496	validation_1-auc:0.80982
[42]	validation_0-auc:0.84440	validation_1-auc:0.80915
[43]	validation_0-auc:0.84486	validation_1-auc:0.80936
[44]	validation_0-auc:0.84481	validation_1-auc:0.80890
[45]	validation_0-auc:0.84452	validation_1-auc:0.80794
[46]	validation_0-auc:0.84496	validation_1-auc:0.80865
[47]	validation_0-auc:0.84539	validation_1-auc:0.80914
[48]	validation_0-auc:0.84568	validation_1-auc:0.80960
[49]	validation_0-auc:0.84631	validation_1-auc:0.81025
[50]	validation_0-auc:0.84594	validation_1-auc:0.80995
[51]	validation_0-auc:0.84645	validation_1-auc:0.81036
[52]	validation_0-auc:0.84601	validation_1-auc:0.80987
[53]	validation_0-auc:0.84568	validation_1-auc:0.80926
[54]	validation_0-auc:0.84550	validation_1-auc:0.80878
[55]	validation_0-auc:0.84532	validation_1-auc:0.80855
[56]	validation_0-auc:0.84579	validation_1-auc:0.80866
[57]	validation_0-auc:0.84616	validation_1-auc:0.80902
[58]	validation_0-auc:0.84662	validation_1-auc:0.80937
[59]	validation_0-auc:0.84657	validation_1-auc:0.80932
[60]	validation_0-auc:0.84697	validation_1-auc:0.80961
[61]	validation_0-auc:0.84658	validation_1-auc:0.80934
[62]	validation_0-auc:0.84630	validation_1-auc:0.80894
[63]	validation_0-auc:0.84680	validation_1-auc:0.80956
[64]	validation_0-auc:0.84643	validation_1-auc:0.80932
[65]	validation_0-auc:0.84679	validation_1-auc:0.80981
[66]	validation_0-auc:0.84731	validation_1-auc:0.81029
[67]	validation_0-auc:0.84766	validation_1-auc:0.81071
[68]	validation_0-auc:0.84806	validation_1-auc:0.81132
[69]	validation_0-auc:0.84837	validation_1-auc:0.81177
[70]	validation_0-auc:0.84879	validation_1-auc:0.81223
[71]	validation_0-auc:0.84898	validation_1-auc:0.81259
[72]	validation_0-auc:0.84926	validation_1-auc:0.81308
[73]	validation_0-auc:0.84905	validation_1-auc:0.81270
[74]	validation_0-auc:0.84940	validation_1-auc:0.81301
[75]	validation_0-auc:0.84900	validation_1-auc:0.81236
[76]	validation_0-auc:0.84870	validation_1-auc:0.81171
[77]	validation_0-auc:0.84892	validation_1-auc:0.81206
[78]	validation_0-auc:0.84922	validation_1-auc:0.81269
[79]	validation_0-auc:0.84910	validation_1-auc:0.81236
[80]	validation_0-auc:0.84885	validation_1-auc:0.81199
[81]	validation_0-auc:0.84857	validation_1-auc:0.81148
[82]	validation_0-auc:0.84835	validation_1-auc:0.81108
[83]	validation_0-auc:0.84863	validation_1-auc:0.81141
[84]	validation_0-auc:0.84860	validation_1-auc:0.81126
[85]	validation_0-auc:0.84909	validation_1-auc:0.81184
[86]	validation_0-auc:0.84942	validation_1-auc:0.81245
[87]	validation_0-auc:0.84927	validation_1-auc:0.81195
[88]	validation_0-auc:0.84927	validation_1-auc:0.81160
[89]	validation_0-auc:0.84966	validation_1-auc:0.81201
[90]	validation_0-auc:0.84997	validation_1-auc:0.81252
[91]	validation_0-auc:0.85031	validation_1-auc:0.81305
[92]	validation_0-auc:0.85002	validation_1-auc:0.81242
[93]	validation_0-auc:0.84990	validation_1-auc:0.81206
[94]	validation_0-auc:0.85013	validation_1-auc:0.81242
[95]	validation_0-auc:0.85011	validation_1-auc:0.81220
[96]	validation_0-auc:0.85056	validation_1-auc:0.81274
[97]	validation_0-auc:0.85091	validation_1-auc:0.81303
[98]	validation_0-auc:0.85123	validation_1-auc:0.81369
[99]	validation_0-auc:0.85110	validation_1-auc:0.81323
[100]	validation_0-auc:0.85145	validation_1-auc:0.81370
[101]	validation_0-auc:0.85170	validation_1-auc:0.81400
[102]	validation_0-auc:0.85192	validation_1-auc:0.81423
[103]	validation_0-auc:0.85225	validation_1-auc:0.81447
[104]	validation_0-auc:0.85231	validation_1-auc:0.81418
[105]	validation_0-auc:0.85231	validation_1-auc:0.81370
[106]	validation_0-auc:0.85225	validation_1-auc:0.81358
[107]	validation_0-auc:0.85241	validation_1-auc:0.81395
[108]	validation_0-auc:0.85242	validation_1-auc:0.81389
[109]	validation_0-auc:0.85261	validation_1-auc:0.81404
[110]	validation_0-auc:0.85239	validation_1-auc:0.81378
[111]	validation_0-auc:0.85260	validation_1-auc:0.81393
[112]	validation_0-auc:0.85279	validation_1-auc:0.81417
[113]	validation_0-auc:0.85267	validation_1-auc:0.81389
[114]	validation_0-auc:0.85257	validation_1-auc:0.81359
[115]	validation_0-auc:0.85277	validation_1-auc:0.81384
[116]	validation_0-auc:0.85267	validation_1-auc:0.81352
[117]	validation_0-auc:0.85267	validation_1-auc:0.81344
[118]	validation_0-auc:0.85292	validation_1-auc:0.81377
[119]	validation_0-auc:0.85319	validation_1-auc:0.81384
[120]	validation_0-auc:0.85341	validation_1-auc:0.81423
[121]	validation_0-auc:0.85362	validation_1-auc:0.81443
[122]	validation_0-auc:0.85383	validation_1-auc:0.81506
[123]	validation_0-auc:0.85369	validation_1-auc:0.81469
[124]	validation_0-auc:0.85390	validation_1-auc:0.81493
[125]	validation_0-auc:0.85385	validation_1-auc:0.81466
[126]	validation_0-auc:0.85373	validation_1-auc:0.81448
[127]	validation_0-auc:0.85374	validation_1-auc:0.81435
[128]	validation_0-auc:0.85366	validation_1-auc:0.81410
[129]	validation_0-auc:0.85392	validation_1-auc:0.81451
[130]	validation_0-auc:0.85391	validation_1-auc:0.81428
[131]	validation_0-auc:0.85378	validation_1-auc:0.81394
[132]	validation_0-auc:0.85398	validation_1-auc:0.81411
[133]	validation_0-auc:0.85430	validation_1-auc:0.81466
[134]	validation_0-auc:0.85447	validation_1-auc:0.81487
[135]	validation_0-auc:0.85448	validation_1-auc:0.81464
[136]	validation_0-auc:0.85443	validation_1-auc:0.81452
[137]	validation_0-auc:0.85437	validation_1-auc:0.81422
[138]	validation_0-auc:0.85451	validation_1-auc:0.81405
[139]	validation_0-auc:0.85474	validation_1-auc:0.81462
[140]	validation_0-auc:0.85471	validation_1-auc:0.81437
[141]	validation_0-auc:0.85465	validation_1-auc:0.81405
[142]	validation_0-auc:0.85500	validation_1-auc:0.81447
[143]	validation_0-auc:0.85497	validation_1-auc:0.81442
[144]	validation_0-auc:0.85497	validation_1-auc:0.81418
[145]	validation_0-auc:0.85492	validation_1-auc:0.81403
[146]	validation_0-auc:0.85521	validation_1-auc:0.81429
[147]	validation_0-auc:0.85555	validation_1-auc:0.81467
[148]	validation_0-auc:0.85572	validation_1-auc:0.81464
[149]	validation_0-auc:0.85592	validation_1-auc:0.81501
[150]	validation_0-auc:0.85585	validation_1-auc:0.81494
[151]	validation_0-auc:0.85620	validation_1-auc:0.81533
[152]	validation_0-auc:0.85617	validation_1-auc:0.81502
[153]	validation_0-auc:0.85632	validation_1-auc:0.81512
[154]	validation_0-auc:0.85630	validation_1-auc:0.81495
[155]	validation_0-auc:0.85670	validation_1-auc:0.81521
[156]	validation_0-auc:0.85718	validation_1-auc:0.81577
[157]	validation_0-auc:0.85708	validation_1-auc:0.81547
[158]	validation_0-auc:0.85740	validation_1-auc:0.81572
[159]	validation_0-auc:0.85733	validation_1-auc:0.81553
[160]	validation_0-auc:0.85745	validation_1-auc:0.81552
[161]	validation_0-auc:0.85741	validation_1-auc:0.81548
[162]	validation_0-auc:0.85783	validation_1-auc:0.81581
[163]	validation_0-auc:0.85792	validation_1-auc:0.81567
[164]	validation_0-auc:0.85827	validation_1-auc:0.81582
[165]	validation_0-auc:0.85864	validation_1-auc:0.81606
[166]	validation_0-auc:0.85896	validation_1-auc:0.81637
[167]	validation_0-auc:0.85894	validation_1-auc:0.81633
[168]	validation_0-auc:0.85935	validation_1-auc:0.81632
[169]	validation_0-auc:0.85968	validation_1-auc:0.81652
[170]	validation_0-auc:0.85996	validation_1-auc:0.81688
[171]	validation_0-auc:0.86023	validation_1-auc:0.81711
[172]	validation_0-auc:0.86044	validation_1-auc:0.81747
[173]	validation_0-auc:0.86076	validation_1-auc:0.81777
[174]	validation_0-auc:0.86079	validation_1-auc:0.81764
[175]	validation_0-auc:0.86105	validation_1-auc:0.81791
[176]	validation_0-auc:0.86138	validation_1-auc:0.81813
[177]	validation_0-auc:0.86144	validation_1-auc:0.81815
[178]	validation_0-auc:0.86151	validation_1-auc:0.81825
[179]	validation_0-auc:0.86193	validation_1-auc:0.81847
[180]	validation_0-auc:0.86201	validation_1-auc:0.81841
[181]	validation_0-auc:0.86193	validation_1-auc:0.81828
[182]	validation_0-auc:0.86195	validation_1-auc:0.81816
[183]	validation_0-auc:0.86207	validation_1-auc:0.81811
[184]	validation_0-auc:0.86228	validation_1-auc:0.81829
[185]	validation_0-auc:0.86233	validation_1-auc:0.81815
[186]	validation_0-auc:0.86238	validation_1-auc:0.81808
[187]	validation_0-auc:0.86264	validation_1-auc:0.81828
[188]	validation_0-auc:0.86268	validation_1-auc:0.81806
[189]	validation_0-auc:0.86301	validation_1-auc:0.81851
[190]	validation_0-auc:0.86316	validation_1-auc:0.81862
[191]	validation_0-auc:0.86324	validation_1-auc:0.81843
[192]	validation_0-auc:0.86351	validation_1-auc:0.81876
[193]	validation_0-auc:0.86361	validation_1-auc:0.81862
[194]	validation_0-auc:0.86362	validation_1-auc:0.81846
[195]	validation_0-auc:0.86394	validation_1-auc:0.81881
[196]	validation_0-auc:0.86396	validation_1-auc:0.81861
[197]	validation_0-auc:0.86405	validation_1-auc:0.81851
[198]	validation_0-auc:0.86415	validation_1-auc:0.81835
[199]	validation_0-auc:0.86444	validation_1-auc:0.81876
[200]	validation_0-auc:0.86447	validation_1-auc:0.81868
[201]	validation_0-auc:0.86479	validation_1-auc:0.81897
[202]	validation_0-auc:0.86501	validation_1-auc:0.81927
[203]	validation_0-auc:0.86521	validation_1-auc:0.81956
[204]	validation_0-auc:0.86530	validation_1-auc:0.81950
[205]	validation_0-auc:0.86534	validation_1-auc:0.81930
[206]	validation_0-auc:0.86553	validation_1-auc:0.81955
[207]	validation_0-auc:0.86555	validation_1-auc:0.81949
[208]	validation_0-auc:0.86569	validation_1-auc:0.81943
[209]	validation_0-auc:0.86573	validation_1-auc:0.81935
[210]	validation_0-auc:0.86576	validation_1-auc:0.81924
[211]	validation_0-auc:0.86580	validation_1-auc:0.81923
[212]	validation_0-auc:0.86608	validation_1-auc:0.81949
[213]	validation_0-auc:0.86611	validation_1-auc:0.81939
[214]	validation_0-auc:0.86612	validation_1-auc:0.81926
[215]	validation_0-auc:0.86638	validation_1-auc:0.81968
[216]	validation_0-auc:0.86654	validation_1-auc:0.82006
[217]	validation_0-auc:0.86675	validation_1-auc:0.82026
[218]	validation_0-auc:0.86701	validation_1-auc:0.82033
[219]	validation_0-auc:0.86721	validation_1-auc:0.82056
[220]	validation_0-auc:0.86747	validation_1-auc:0.82077
[221]	validation_0-auc:0.86759	validation_1-auc:0.82108
[222]	validation_0-auc:0.86767	validation_1-auc:0.82094
[223]	validation_0-auc:0.86781	validation_1-auc:0.82110
[224]	validation_0-auc:0.86782	validation_1-auc:0.82108
[225]	validation_0-auc:0.86791	validation_1-auc:0.82103
[226]	validation_0-auc:0.86796	validation_1-auc:0.82095
[227]	validation_0-auc:0.86802	validation_1-auc:0.82091
[228]	validation_0-auc:0.86806	validation_1-auc:0.82080
[229]	validation_0-auc:0.86810	validation_1-auc:0.82081
[230]	validation_0-auc:0.86813	validation_1-auc:0.82084
[231]	validation_0-auc:0.86828	validation_1-auc:0.82105
[232]	validation_0-auc:0.86848	validation_1-auc:0.82131
[233]	validation_0-auc:0.86861	validation_1-auc:0.82137
[234]	validation_0-auc:0.86872	validation_1-auc:0.82128
[235]	validation_0-auc:0.86881	validation_1-auc:0.82123
[236]	validation_0-auc:0.86890	validation_1-auc:0.82120
[237]	validation_0-auc:0.86907	validation_1-auc:0.82137
[238]	validation_0-auc:0.86914	validation_1-auc:0.82129
[239]	validation_0-auc:0.86930	validation_1-auc:0.82159
[240]	validation_0-auc:0.86942	validation_1-auc:0.82179
[241]	validation_0-auc:0.86956	validation_1-auc:0.82202
[242]	validation_0-auc:0.86965	validation_1-auc:0.82201
[243]	validation_0-auc:0.86974	validation_1-auc:0.82197
[244]	validation_0-auc:0.86992	validation_1-auc:0.82209
[245]	validation_0-auc:0.86995	validation_1-auc:0.82205
[246]	validation_0-auc:0.87012	validation_1-auc:0.82222
[247]	validation_0-auc:0.87032	validation_1-auc:0.82235
[248]	validation_0-auc:0.87039	validation_1-auc:0.82241
[249]	validation_0-auc:0.87058	validation_1-auc:0.82261
[250]	validation_0-auc:0.87066	validation_1-auc:0.82259
[251]	validation_0-auc:0.87074	validation_1-auc:0.82266
[252]	validation_0-auc:0.87085	validation_1-auc:0.82286
[253]	validation_0-auc:0.87097	validation_1-auc:0.82299
[254]	validation_0-auc:0.87102	validation_1-auc:0.82294
[255]	validation_0-auc:0.87118	validation_1-auc:0.82306
[256]	validation_0-auc:0.87124	validation_1-auc:0.82303
[257]	validation_0-auc:0.87125	validation_1-auc:0.82301
[258]	validation_0-auc:0.87140	validation_1-auc:0.82325
[259]	validation_0-auc:0.87155	validation_1-auc:0.82337
[260]	validation_0-auc:0.87165	validation_1-auc:0.82358
[261]	validation_0-auc:0.87178	validation_1-auc:0.82365
[262]	validation_0-auc:0.87190	validation_1-auc:0.82371
[263]	validation_0-auc:0.87200	validation_1-auc:0.82385
[264]	validation_0-auc:0.87211	validation_1-auc:0.82396
[265]	validation_0-auc:0.87222	validation_1-auc:0.82410
[266]	validation_0-auc:0.87224	validation_1-auc:0.82410
[267]	validation_0-auc:0.87233	validation_1-auc:0.82404
[268]	validation_0-auc:0.87240	validation_1-auc:0.82409
[269]	validation_0-auc:0.87245	validation_1-auc:0.82409
[270]	validation_0-auc:0.87253	validation_1-auc:0.82420
[271]	validation_0-auc:0.87261	validation_1-auc:0.82429
[272]	validation_0-auc:0.87269	validation_1-auc:0.82438
[273]	validation_0-auc:0.87279	validation_1-auc:0.82432
[274]	validation_0-auc:0.87288	validation_1-auc:0.82433
[275]	validation_0-auc:0.87296	validation_1-auc:0.82430
[276]	validation_0-auc:0.87305	validation_1-auc:0.82427
[277]	validation_0-auc:0.87312	validation_1-auc:0.82434
[278]	validation_0-auc:0.87325	validation_1-auc:0.82436
[279]	validation_0-auc:0.87331	validation_1-auc:0.82435
[280]	validation_0-auc:0.87339	validation_1-auc:0.82439
[281]	validation_0-auc:0.87348	validation_1-auc:0.82440
[282]	validation_0-auc:0.87354	validation_1-auc:0.82448
[283]	validation_0-auc:0.87363	validation_1-auc:0.82459
[284]	validation_0-auc:0.87373	validation_1-auc:0.82469
[285]	validation_0-auc:0.87383	validation_1-auc:0.82475
[286]	validation_0-auc:0.87394	validation_1-auc:0.82476
[287]	validation_0-auc:0.87404	validation_1-auc:0.82487
[288]	validation_0-auc:0.87409	validation_1-auc:0.82482
[289]	validation_0-auc:0.87418	validation_1-auc:0.82492
[290]	validation_0-auc:0.87423	validation_1-auc:0.82490
[291]	validation_0-auc:0.87430	validation_1-auc:0.82494
[292]	validation_0-auc:0.87442	validation_1-auc:0.82493
[293]	validation_0-auc:0.87446	validation_1-auc:0.82496
[294]	validation_0-auc:0.87456	validation_1-auc:0.82497
[295]	validation_0-auc:0.87459	validation_1-auc:0.82494
[296]	validation_0-auc:0.87466	validation_1-auc:0.82503
[297]	validation_0-auc:0.87473	validation_1-auc:0.82504
[298]	validation_0-auc:0.87477	validation_1-auc:0.82507
[299]	validation_0-auc:0.87486	validation_1-auc:0.82518
[300]	validation_0-auc:0.87493	validation_1-auc:0.82520
[301]	validation_0-auc:0.87505	validation_1-auc:0.82515
[302]	validation_0-auc:0.87514	validation_1-auc:0.82519
[303]	validation_0-auc:0.87518	validation_1-auc:0.82523
[304]	validation_0-auc:0.87528	validation_1-auc:0.82527
[305]	validation_0-auc:0.87531	validation_1-auc:0.82533
[306]	validation_0-auc:0.87543	validation_1-auc:0.82540
[307]	validation_0-auc:0.87551	validation_1-auc:0.82544
[308]	validation_0-auc:0.87555	validation_1-auc:0.82542
[309]	validation_0-auc:0.87562	validation_1-auc:0.82549
[310]	validation_0-auc:0.87566	validation_1-auc:0.82554
[311]	validation_0-auc:0.87573	validation_1-auc:0.82556
[312]	validation_0-auc:0.87588	validation_1-auc:0.82557
[313]	validation_0-auc:0.87599	validation_1-auc:0.82554
[314]	validation_0-auc:0.87602	validation_1-auc:0.82562
[315]	validation_0-auc:0.87607	validation_1-auc:0.82565
[316]	validation_0-auc:0.87611	validation_1-auc:0.82572
[317]	validation_0-auc:0.87614	validation_1-auc:0.82568
[318]	validation_0-auc:0.87625	validation_1-auc:0.82566
[319]	validation_0-auc:0.87628	validation_1-auc:0.82563
[320]	validation_0-auc:0.87633	validation_1-auc:0.82569
[321]	validation_0-auc:0.87646	validation_1-auc:0.82570
[322]	validation_0-auc:0.87650	validation_1-auc:0.82574
[323]	validation_0-auc:0.87657	validation_1-auc:0.82585
[324]	validation_0-auc:0.87664	validation_1-auc:0.82595
[325]	validation_0-auc:0.87667	validation_1-auc:0.82593
[326]	validation_0-auc:0.87674	validation_1-auc:0.82600
[327]	validation_0-auc:0.87677	validation_1-auc:0.82603
[328]	validation_0-auc:0.87682	validation_1-auc:0.82603
[329]	validation_0-auc:0.87686	validation_1-auc:0.82602
[330]	validation_0-auc:0.87689	validation_1-auc:0.82606
[331]	validation_0-auc:0.87692	validation_1-auc:0.82608
[332]	validation_0-auc:0.87696	validation_1-auc:0.82617
[333]	validation_0-auc:0.87698	validation_1-auc:0.82616
[334]	validation_0-auc:0.87703	validation_1-auc:0.82613
[335]	validation_0-auc:0.87709	validation_1-auc:0.82617
[336]	validation_0-auc:0.87715	validation_1-auc:0.82620
[337]	validation_0-auc:0.87717	validation_1-auc:0.82625
[338]	validation_0-auc:0.87720	validation_1-auc:0.82631
[339]	validation_0-auc:0.87723	validation_1-auc:0.82636
[340]	validation_0-auc:0.87736	validation_1-auc:0.82634
[341]	validation_0-auc:0.87739	validation_1-auc:0.82630
[342]	validation_0-auc:0.87752	validation_1-auc:0.82629
[343]	validation_0-auc:0.87755	validation_1-auc:0.82632
[344]	validation_0-auc:0.87760	validation_1-auc:0.82630
[345]	validation_0-auc:0.87766	validation_1-auc:0.82628
[346]	validation_0-auc:0.87770	validation_1-auc:0.82630
[347]	validation_0-auc:0.87773	validation_1-auc:0.82631
[348]	validation_0-auc:0.87776	validation_1-auc:0.82630
[349]	validation_0-auc:0.87783	validation_1-auc:0.82633
[350]	validation_0-auc:0.87787	validation_1-auc:0.82636
[351]	validation_0-auc:0.87788	validation_1-auc:0.82637
[352]	validation_0-auc:0.87802	validation_1-auc:0.82638
[353]	validation_0-auc:0.87805	validation_1-auc:0.82644
[354]	validation_0-auc:0.87811	validation_1-auc:0.82647
[355]	validation_0-auc:0.87815	validation_1-auc:0.82646
[356]	validation_0-auc:0.87828	validation_1-auc:0.82642
[357]	validation_0-auc:0.87834	validation_1-auc:0.82639
[358]	validation_0-auc:0.87837	validation_1-auc:0.82636
[359]	validation_0-auc:0.87839	validation_1-auc:0.82635
[360]	validation_0-auc:0.87842	validation_1-auc:0.82636
[361]	validation_0-auc:0.87847	validation_1-auc:0.82647
[362]	validation_0-auc:0.87849	validation_1-auc:0.82644
[363]	validation_0-auc:0.87854	validation_1-auc:0.82644
[364]	validation_0-auc:0.87860	validation_1-auc:0.82643
[365]	validation_0-auc:0.87864	validation_1-auc:0.82651
[366]	validation_0-auc:0.87874	validation_1-auc:0.82647
[367]	validation_0-auc:0.87875	validation_1-auc:0.82648
[368]	validation_0-auc:0.87881	validation_1-auc:0.82644
[369]	validation_0-auc:0.87888	validation_1-auc:0.82646
[370]	validation_0-auc:0.87891	validation_1-auc:0.82645
[371]	validation_0-auc:0.87894	validation_1-auc:0.82646
[372]	validation_0-auc:0.87902	validation_1-auc:0.82649
[373]	validation_0-auc:0.87907	validation_1-auc:0.82650
[374]	validation_0-auc:0.87916	validation_1-auc:0.82648
[375]	validation_0-auc:0.87919	validation_1-auc:0.82648
[376]	validation_0-auc:0.87921	validation_1-auc:0.82651
[377]	validation_0-auc:0.87925	validation_1-auc:0.82653
[378]	validation_0-auc:0.87927	validation_1-auc:0.82653
[379]	validation_0-auc:0.87928	validation_1-auc:0.82653
[380]	validation_0-auc:0.87938	validation_1-auc:0.82649
[381]	validation_0-auc:0.87947	validation_1-auc:0.82650
[382]	validation_0-auc:0.87949	validation_1-auc:0.82651
[383]	validation_0-auc:0.87957	validation_1-auc:0.82654
[384]	validation_0-auc:0.87959	validation_1-auc:0.82654
[385]	validation_0-auc:0.87961	validation_1-auc:0.82654
[386]	validation_0-auc:0.87963	validation_1-auc:0.82652
[387]	validation_0-auc:0.87970	validation_1-auc:0.82656
[388]	validation_0-auc:0.87973	validation_1-auc:0.82655
[389]	validation_0-auc:0.87978	validation_1-auc:0.82657
[390]	validation_0-auc:0.87980	validation_1-auc:0.82660
[391]	validation_0-auc:0.87986	validation_1-auc:0.82659
[392]	validation_0-auc:0.87988	validation_1-auc:0.82660
[393]	validation_0-auc:0.87992	validation_1-auc:0.82659
[394]	validation_0-auc:0.87996	validation_1-auc:0.82656
[395]	validation_0-auc:0.87998	validation_1-auc:0.82657
[396]	validation_0-auc:0.88003	validation_1-auc:0.82657
[397]	validation_0-auc:0.88005	validation_1-auc:0.82655
[398]	validation_0-auc:0.88017	validation_1-auc:0.82653
[399]	validation_0-auc:0.88019	validation_1-auc:0.82650
[400]	validation_0-auc:0.88022	validation_1-auc:0.82649
[401]	validation_0-auc:0.88028	validation_1-auc:0.82652
[402]	validation_0-auc:0.88034	validation_1-auc:0.82656
[403]	validation_0-auc:0.88041	validation_1-auc:0.82654
[404]	validation_0-auc:0.88044	validation_1-auc:0.82654
[405]	validation_0-auc:0.88047	validation_1-auc:0.82651
[406]	validation_0-auc:0.88051	validation_1-auc:0.82646
[407]	validation_0-auc:0.88051	validation_1-auc:0.82645
[408]	validation_0-auc:0.88061	validation_1-auc:0.82646
[409]	validation_0-auc:0.88069	validation_1-auc:0.82648
[410]	validation_0-auc:0.88071	validation_1-auc:0.82645
[411]	validation_0-auc:0.88074	validation_1-auc:0.82642
[412]	validation_0-auc:0.88081	validation_1-auc:0.82637
[413]	validation_0-auc:0.88082	validation_1-auc:0.82637
[414]	validation_0-auc:0.88084	validation_1-auc:0.82634
[415]	validation_0-auc:0.88086	validation_1-auc:0.82634
[416]	validation_0-auc:0.88087	validation_1-auc:0.82635
[417]	validation_0-auc:0.88096	validation_1-auc:0.82632
[418]	validation_0-auc:0.88103	validation_1-auc:0.82636
[419]	validation_0-auc:0.88108	validation_1-auc:0.82638
[420]	validation_0-auc:0.88115	validation_1-auc:0.82640
[421]	validation_0-auc:0.88117	validation_1-auc:0.82639
[422]	validation_0-auc:0.88118	validation_1-auc:0.82639
[423]	validation_0-auc:0.88121	validation_1-auc:0.82634
[424]	validation_0-auc:0.88125	validation_1-auc:0.82640
[425]	validation_0-auc:0.88128	validation_1-auc:0.82638
[426]	validation_0-auc:0.88135	validation_1-auc:0.82636
[427]	validation_0-auc:0.88140	validation_1-auc:0.82636
[428]	validation_0-auc:0.88143	validation_1-auc:0.82634
[429]	validation_0-auc:0.88146	validation_1-auc:0.82635
[430]	validation_0-auc:0.88148	validation_1-auc:0.82633
[431]	validation_0-auc:0.88149	validation_1-auc:0.82635
[432]	validation_0-auc:0.88150	validation_1-auc:0.82635
[433]	validation_0-auc:0.88154	validation_1-auc:0.82636
[434]	validation_0-auc:0.88162	validation_1-auc:0.82634
[435]	validation_0-auc:0.88166	validation_1-auc:0.82637
[436]	validation_0-auc:0.88168	validation_1-auc:0.82636
[437]	validation_0-auc:0.88171	validation_1-auc:0.82634
[438]	validation_0-auc:0.88174	validation_1-auc:0.82635
[439]	validation_0-auc:0.88178	validation_1-auc:0.82631
[440]	validation_0-auc:0.88185	validation_1-auc:0.82635
[441]	validation_0-auc:0.88186	validation_1-auc:0.82635
[442]	validation_0-auc:0.88187	validation_1-auc:0.82634
[443]	validation_0-auc:0.88191	validation_1-auc:0.82632
[444]	validation_0-auc:0.88196	validation_1-auc:0.82636
[445]	validation_0-auc:0.88205	validation_1-auc:0.82635
[446]	validation_0-auc:0.88209	validation_1-auc:0.82633
[447]	validation_0-auc:0.88212	validation_1-auc:0.82630
[448]	validation_0-auc:0.88216	validation_1-auc:0.82628
[449]	validation_0-auc:0.88225	validation_1-auc:0.82626
[450]	validation_0-auc:0.88225	validation_1-auc:0.82625
[451]	validation_0-auc:0.88231	validation_1-auc:0.82626
[452]	validation_0-auc:0.88233	validation_1-auc:0.82624
[453]	validation_0-auc:0.88235	validation_1-auc:0.82625
[454]	validation_0-auc:0.88242	validation_1-auc:0.82630
[455]	validation_0-auc:0.88247	validation_1-auc:0.82627
[456]	validation_0-auc:0.88249	validation_1-auc:0.82625
[457]	validation_0-auc:0.88256	validation_1-auc:0.82622
[458]	validation_0-auc:0.88257	validation_1-auc:0.82619
[459]	validation_0-auc:0.88258	validation_1-auc:0.82618
[460]	validation_0-auc:0.88261	validation_1-auc:0.82618
[461]	validation_0-auc:0.88262	validation_1-auc:0.82615
[462]	validation_0-auc:0.88264	validation_1-auc:0.82615
[463]	validation_0-auc:0.88267	validation_1-auc:0.82618
[464]	validation_0-auc:0.88269	validation_1-auc:0.82615
[465]	validation_0-auc:0.88271	validation_1-auc:0.82613
[466]	validation_0-auc:0.88274	validation_1-auc:0.82613
[467]	validation_0-auc:0.88276	validation_1-auc:0.82611
[468]	validation_0-auc:0.88279	validation_1-auc:0.82610
[469]	validation_0-auc:0.88284	validation_1-auc:0.82607
[470]	validation_0-auc:0.88285	validation_1-auc:0.82605
[471]	validation_0-auc:0.88287	validation_1-auc:0.82603
[472]	validation_0-auc:0.88290	validation_1-auc:0.82602
[473]	validation_0-auc:0.88294	validation_1-auc:0.82602
[474]	validation_0-auc:0.88300	validation_1-auc:0.82601
[475]	validation_0-auc:0.88315	validation_1-auc:0.82610
[476]	validation_0-auc:0.88318	validation_1-auc:0.82609
[477]	validation_0-auc:0.88324	validation_1-auc:0.82613
[478]	validation_0-auc:0.88326	validation_1-auc:0.82613
[479]	validation_0-auc:0.88332	validation_1-auc:0.82611
[480]	validation_0-auc:0.88335	validation_1-auc:0.82608
[481]	validation_0-auc:0.88337	validation_1-auc:0.82607
[482]	validation_0-auc:0.88343	validation_1-auc:0.82604
[483]	validation_0-auc:0.88344	validation_1-auc:0.82606
[484]	validation_0-auc:0.88351	validation_1-auc:0.82606
[485]	validation_0-auc:0.88357	validation_1-auc:0.82607
[486]	validation_0-auc:0.88366	validation_1-auc:0.82607
[487]	validation_0-auc:0.88369	validation_1-auc:0.82607
[488]	validation_0-auc:0.88373	validation_1-auc:0.82610
[489]	validation_0-auc:0.88376	validation_1-auc:0.82612
[490]	validation_0-auc:0.88381	validation_1-auc:0.82610
[491]	validation_0-auc:0.88385	validation_1-auc:0.82614
[492]	validation_0-auc:0.88390	validation_1-auc:0.82614
[493]	validation_0-auc:0.88395	validation_1-auc:0.82613
[494]	validation_0-auc:0.88399	validation_1-auc:0.82615
[495]	validation_0-auc:0.88400	validation_1-auc:0.82613
[496]	validation_0-auc:0.88404	validation_1-auc:0.82615
[497]	validation_0-auc:0.88406	validation_1-auc:0.82612
[498]	validation_0-auc:0.88407	validation_1-auc:0.82612
[499]	validation_0-auc:0.88413	validation_1-auc:0.82616
[500]	validation_0-auc:0.88415	validation_1-auc:0.82616
[501]	validation_0-auc:0.88417	validation_1-auc:0.82613
[502]	validation_0-auc:0.88422	validation_1-auc:0.82614
[503]	validation_0-auc:0.88423	validation_1-auc:0.82614
[504]	validation_0-auc:0.88425	validation_1-auc:0.82611
[505]	validation_0-auc:0.88430	validation_1-auc:0.82607
[506]	validation_0-auc:0.88432	validation_1-auc:0.82606
[507]	validation_0-auc:0.88436	validation_1-auc:0.82605
[508]	validation_0-auc:0.88444	validation_1-auc:0.82602
[509]	validation_0-auc:0.88445	validation_1-auc:0.82602
[510]	validation_0-auc:0.88448	validation_1-auc:0.82602
[511]	validation_0-auc:0.88456	validation_1-auc:0.82599
[512]	validation_0-auc:0.88461	validation_1-auc:0.82598
[513]	validation_0-auc:0.88462	validation_1-auc:0.82596
[514]	validation_0-auc:0.88468	validation_1-auc:0.82593
[515]	validation_0-auc:0.88469	validation_1-auc:0.82592
[516]	validation_0-auc:0.88471	validation_1-auc:0.82590
[517]	validation_0-auc:0.88477	validation_1-auc:0.82591
[518]	validation_0-auc:0.88481	validation_1-auc:0.82589
[519]	validation_0-auc:0.88483	validation_1-auc:0.82589
[520]	validation_0-auc:0.88484	validation_1-auc:0.82587
[521]	validation_0-auc:0.88485	validation_1-auc:0.82586
[522]	validation_0-auc:0.88487	validation_1-auc:0.82583
[523]	validation_0-auc:0.88488	validation_1-auc:0.82584
[524]	validation_0-auc:0.88494	validation_1-auc:0.82584
[525]	validation_0-auc:0.88495	validation_1-auc:0.82584
[526]	validation_0-auc:0.88502	validation_1-auc:0.82582
[527]	validation_0-auc:0.88512	validation_1-auc:0.82586
[528]	validation_0-auc:0.88522	validation_1-auc:0.82591
[529]	validation_0-auc:0.88522	validation_1-auc:0.82590
[530]	validation_0-auc:0.88526	validation_1-auc:0.82588
[531]	validation_0-auc:0.88530	validation_1-auc:0.82588
[532]	validation_0-auc:0.88532	validation_1-auc:0.82585
[533]	validation_0-auc:0.88537	validation_1-auc:0.82584
[534]	validation_0-auc:0.88538	validation_1-auc:0.82584
[535]	validation_0-auc:0.88540	validation_1-auc:0.82583
[536]	validation_0-auc:0.88542	validation_1-auc:0.82582
[537]	validation_0-auc:0.88545	validation_1-auc:0.82580
[538]	validation_0-auc:0.88553	validation_1-auc:0.82584
[539]	validation_0-auc:0.88560	validation_1-auc:0.82585
[540]	validation_0-auc:0.88566	validation_1-auc:0.82586
[541]	validation_0-auc:0.88572	validation_1-auc:0.82584
[542]	validation_0-auc:0.88574	validation_1-auc:0.82586
[543]	validation_0-auc:0.88578	validation_1-auc:0.82582
[544]	validation_0-auc:0.88582	validation_1-auc:0.82582
[545]	validation_0-auc:0.88585	validation_1-auc:0.82580
[546]	validation_0-auc:0.88587	validation_1-auc:0.82578
[547]	validation_0-auc:0.88590	validation_1-auc:0.82578
[548]	validation_0-auc:0.88594	validation_1-auc:0.82581
[549]	validation_0-auc:0.88598	validation_1-auc:0.82580
[550]	validation_0-auc:0.88602	validation_1-auc:0.82577
[551]	validation_0-auc:0.88603	validation_1-auc:0.82577
[552]	validation_0-auc:0.88607	validation_1-auc:0.82577
[553]	validation_0-auc:0.88614	validation_1-auc:0.82574
[554]	validation_0-auc:0.88615	validation_1-auc:0.82574
[555]	validation_0-auc:0.88618	validation_1-auc:0.82570
[556]	validation_0-auc:0.88626	validation_1-auc:0.82579
[557]	validation_0-auc:0.88631	validation_1-auc:0.82575
[558]	validation_0-auc:0.88632	validation_1-auc:0.82575
[559]	validation_0-auc:0.88633	validation_1-auc:0.82575
[560]	validation_0-auc:0.88634	validation_1-auc:0.82574
[561]	validation_0-auc:0.88641	validation_1-auc:0.82578
[562]	validation_0-auc:0.88645	validation_1-auc:0.82579
[563]	validation_0-auc:0.88647	validation_1-auc:0.82577
[564]	validation_0-auc:0.88649	validation_1-auc:0.82577
[565]	validation_0-auc:0.88651	validation_1-auc:0.82576
[566]	validation_0-auc:0.88654	validation_1-auc:0.82575
[567]	validation_0-auc:0.88656	validation_1-auc:0.82572
[568]	validation_0-auc:0.88658	validation_1-auc:0.82570
[569]	validation_0-auc:0.88659	validation_1-auc:0.82571
[570]	validation_0-auc:0.88663	validation_1-auc:0.82571
[571]	validation_0-auc:0.88663	validation_1-auc:0.82571
[572]	validation_0-auc:0.88666	validation_1-auc:0.82571
[573]	validation_0-auc:0.88669	validation_1-auc:0.82567
[574]	validation_0-auc:0.88675	validation_1-auc:0.82564
[575]	validation_0-auc:0.88683	validation_1-auc:0.82565
[576]	validation_0-auc:0.88686	validation_1-auc:0.82567
[577]	validation_0-auc:0.88687	validation_1-auc:0.82564
[578]	validation_0-auc:0.88690	validation_1-auc:0.82561
[579]	validation_0-auc:0.88692	validation_1-auc:0.82560
[580]	validation_0-auc:0.88694	validation_1-auc:0.82559
[581]	validation_0-auc:0.88696	validation_1-auc:0.82558
[582]	validation_0-auc:0.88699	validation_1-auc:0.82556
[583]	validation_0-auc:0.88705	validation_1-auc:0.82555
[584]	validation_0-auc:0.88709	validation_1-auc:0.82554
[585]	validation_0-auc:0.88710	validation_1-auc:0.82554
[586]	validation_0-auc:0.88711	validation_1-auc:0.82553
[587]	validation_0-auc:0.88713	validation_1-auc:0.82552
[588]	validation_0-auc:0.88714	validation_1-auc:0.82553
[589]	validation_0-auc:0.88715	validation_1-auc:0.82551
[590]	validation_0-auc:0.88716	validation_1-auc:0.82551
[591]	validation_0-auc:0.88718	validation_1-auc:0.82549
[592]	validation_0-auc:0.88724	validation_1-auc:0.82545
CPU times: user 15min 35s, sys: 3.45 s, total: 15min 39s
Wall time: 4min 16s

XGBClassifier(base_score=0.5, booster='gbtree', callbacks=None,
              colsample_bylevel=1, colsample_bynode=1, colsample_bytree=0.5,
              early_stopping_rounds=None, enable_categorical=False,
              eval_metric=None, gamma=0, gpu_id=-1, grow_policy='depthwise',
              importance_type=None, interaction_constraints='',
              learning_rate=0.02, max_bin=256, max_cat_to_onehot=4,
              max_delta_step=0, max_depth=5, max_leaves=0, min_child_weight=3,
              missing=nan, monotone_constraints='()', n_estimators=1000,
              n_jobs=0, num_parallel_tree=1, predictor='auto', random_state=77,
              reg_alpha=0.03, reg_lambda=1, ...)

pred_prob = xgb_model_l.predict_proba(X_test)[:, 1]
xgb_roc_score = roc_auc_score(y_test, pred_prob, average='macro')
print("ROC AUC : {0:4f}".format(xgb_roc_score))

ROC AUC : 0.826601

메모

• XGBoost는 GBM을 기반으로 하고 있기에, 수행시간이 어느정도 걸립니다.
• 앙상블 계열 알고리즘에서 하이퍼 파라미터 튜닝으로 성능 수치 개선이 급격하게 되는 경우는 많지 않습니다.

각 특징의 중요도 시각화

• xgboost 모듈의 시각화 기능을 갖는 plot_importance() 메서드를 이용

from xgboost import plot_importance
import matplotlib.pyplot as plt

fig, ax = plt.subplots(1,1, figsize=(10,8))
plot_importance(xgb_model_l, ax=ax, max_num_features=20, height=0.4)

<AxesSubplot:title={'center':'Feature importance'}, xlabel='F score', ylabel='Features'>

• var38, var15, saldo_var30 등이 유의한 변수 TOP3로 선정