Computer scienceData scienceMachine learningIntroduction to deep learning

Activation functions

Too many choices

Report a typo

What are some general guidelines for using different activation functions?

Hint

Review the Some considerations for using different activation functions section of the theory.

Select one or more options from the list

RELU is a standard choice, but it's helpful to monitor the number of dead neurons

It is common to mix different activation functions for the hidden layers within a single architecture

With any activation function, it's helpful to monitor the number of dead neurons and the learning rates

Tahn will almost always outperform RELU

Sigmoid should never be used in practice as an activation function for the hidden layers

___

Create a free account to access the full topic