[Advanced Learning Algorithms] Neural network training - Neural Network Training

 TensorFlow implementation

 

[Train a Neural Network in TensorFlow]

 

import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

# step 1) specify the model
model = Sequential([
    Dense(units=25, activation='sigmoid'),
    Dense(units=15, activation='sigmoid'),
    Dense(units=1, activation='sigmoid')
])

from tensorflow.keras.losses import BinaryCrossentropy

# step 2) compile model using a specific loss function 
model.compile(loss=BinaryCrossentropy())

# step 3) train the model
model.fit(X,Y,epochs=100)

 

 * epochs: Gradient descent를 시행하는 횟수

 

 

 

 Training Details

 

[Model Training Steps]

 

1. Create the model

 주어진 input x, w, b 값으로 output 값 계산을 어떻게 할 지에 대한 model을 만들어야 한다. 예를 들면 logistic regression이라면 sigmoid 함수를 사용하고, regression이라면 linear regression model을 사용해야 한다. 

model = Sequential([
    Dense(units=25, activation='sigmoid'),
    Dense(units=15, activation='sigmoid'),
    Dense(units=1, activation='sigmoid')
])

 

2. Loss and cost functions

위의 model에 따라 loss function의 형태도 달라지는데, 이에 맞추어 loss function을 설정해야 한다. 

# logistic loss
from tensorflow.keras.losses import BinaryCrossentropy
model.compile(loss=BinaryCrossentropy())

# linear loss
from tensorflow.keras.losses import MeanSquaredError
model.compile(loss=MeanSquaredError())

 

3. Gradient descent

 기존의 data에 맞추어 cost function을 최소화할 수 있도록 gradient descent를 수행해야 한다. 

 

model.fit(X,Y,epochs=100)