lecture3 preview, keyword

1.Loss Function

 

multiclass svm loss

data loss / regularization / occam's Razor

L1/L2 Regularization

softmax classifier(Multinomial Logistic Regression)

unnormalized probabilities

matrix multiply + bias effect / hinge loss(svm), cross-entropy loss(softmax)

softmax vs SVM

Recap - score function / loss function

 

2.Optimization

 

1)random search

2)follow the slope - derivative, calculus, analytic gradient

numerical gradient vs analytic gradient

stocastic gradient descent(SGD) - mini batch

image features - motivation,

  feature transform,

  color histogram, 

  histogram of oriented gradients(HoG)

  Bag of Words

Image features vs ConvNets

  feature extraction