Why is sigmoid activation function not recommended for hidden units but is fine for an output...

60.1K

Verified Solution

Question

Basic Math

Why is sigmoid activation function not recommended for hiddenunits but is fine for an output unit?

Answer & Explanation Solved by verified expert

3.9 Ratings (676 Votes)

Answer As you can see the gradient for the sigmoid function will saturate and when using the chain rule it will contract By difference the subsidiary for ReLU is dependably 1 or 0 The sigmoid Gaussian and sinusoidal capacities are chosen because of their autonomous and major space division properties The sigmoid capacity isnt successful for a solitary concealed unit Despite what might be expected alternate capacities can give great execution At the point when a few shrouded units are utilized the sigmoid capacity is helpful Be that as it may the union speed is still slower than the others The Gaussian function is sensitive to the additive noise while the others are rather insensitive As a result See Answer

Get Answers to Unlimited Questions

Join us to gain access to millions of questions and expert answers. Enjoy exclusive benefits tailored just for you!

Membership Benefits:

Unlimited Question Access with detailed Answers
Zin AI - 3 Million Words
10 Dall-E 3 Images
20 Plot Generations
Conversation with Dialogue Memory
No Ads, Ever!
Access to Our Best AI Platform: Flex AI - Your personal assistant for all your inquiries!

Become a Member