Which of the following machine learning algorithms is unsupervised?

random forest

Although it can be used to as a dissimilarity measure between unlabeled data, random forest is mostly used in supervised learning for classification, regression, and other tasks.

k-nearest neighbors

k-nearest neighbors is a non-parametric method used for classification and regression. It is supervised because it compares a given input to labeled training examples to predict an output.

k-means

During training, k-means partitions observations into k clusters. During inference, it assigns a given data point to the nearest cluster by distance. k-means is unsupervised, because it doesn't require labeled data to be trained.

support-vector machines

Support-vector machines (SVMs) are supervised learning models. Given a labeled training set, SVMs constructs a hyperplane or set of hyperplanes separating the data. There is a similar method to SVM called support-vector clustering (SVC) built for unsupervised learning.

Submit

Averaging the output of multiple decision trees helps to$:$

increase bias

No, the bias is roughly the same for the average and individual decision trees. Averaging simply smoothes the output of multiple decision trees trained on the same data.

increase variance

No. The average of models is less sensitive to small changes in the training set than individual models, so the variance doesn't increase.

decrease bias

No, the bias is roughly the same for the average and individual decision trees. Averaging simply smoothes the output of multiple decision trees trained on the same data.

decrease variance

Yes. Averaging models leads to higher stability and a lower variance than individual models. Mathematically, remember that $Var(\bar{X})=\frac{Var(X)}{N}

Submit

To optimize your objective function, you are performing full batch gradient descent using the entire training set (not stochastic gradient descent).

Is it required to shuffle your training set?

Yes. If you don't, the optimization will oscillate around the minimum at the end of training.

The learning rate might impact the oscillations of the loss value during training, but shuffling won't.

Yes, in order to help the model generalize to the test dataset.

Shuffling the data won't have any impact on generalization in this scenario.

No, it is not necessary because the dataset can already be considered shuffled from the data collection process.

It is true that shuffling isn't necessary in this scenario, but the reason is different from the one stated above.

No, because each update passes through the entire dataset anyway and the order doesn't matter.

Correct. At every iteration, full batch gradient descent uses the entire training set to compute a gradient. The order in which data is processed doesn't impact the gradient value.

Submit

You've received 1,000,000 images and have split it in 96%/2%/2% between train, dev and test sets. You've trained your model, and analyzed the results. After working further on the problem, you’ve decided to correct the incorrectly labeled data on the dev set.

Which of these statements do you agree with?

You should also correct the incorrectly labeled data in the test set, so that the dev and test sets still come from the same distribution.

Yes. It is important that your dev and test set have the closest possible distribution to "real" data.

You should correct incorrectly labeled data in the training set as well so as to avoid your training set now being even more different from your dev set.

It is also important for the training set to avoid having a data mismatch problem. However, the first option will have a higher return on investment given how large the training set is compared to the dev and test sets.

You should not correct the incorrectly labeled data in the test set, because the test set should reflect the data distribution of the real world.

No, mised data does not mirror the data distribution of the real world.

If you want to correct incorrectly labeled data, you should do it on all three sets (train/dev/test) in order to maintain similar distributions.

Although this statement seem correct, it doesn't have the same return on investment as other propositions. In a machine learning project, you should prioritize tasks that will benefit you most.

Submit

You're working on a binary classification task, to classify if an image contains a cat ("1") or doesn't contain a cat ("0").

What loss would you choose to minimize in order to train a model?

$\mathcal{L}= y\log\hat{y}+(1-y)\log(1-\hat{y})$

Remember that you are trying to minimize the loss function.

$\mathcal{L}= -y\log\hat{y}-(1-y)\log(1-\hat{y})$

Good job. You are trying to minimize the binary cross entropy loss over the training set.

$\mathcal{L}= \lVert {y - \hat{y}} \rVert^2_{2}$

For binary classification tasks, it is more common to use the binary cross entropy loss rather than the L2 loss.

$\mathcal{L}= \lVert {y - \hat{y}} \rVert^2_{2}$ + constant

For binary classification tasks, it is more common to use the binary cross entropy loss rather than the L2 loss.

Submit