CS 5354/CS 4365 Intelligent Computing
Fall 2019, Test 2

Name: ____________________________

General comments:

Good luck!

1. Use Lagrange multiplier method to solve the following constraint optimization problem: find a point on an ellipse x12 + 2x22 = 1 which is the furthest away from the point (4, 3).

2. Perform the first step of gradient descent to find the minimum of a function 2(x1 + 2)2 + 3(x2 − 2)2. Start with a point (0, 0) and use α = 0.1.

3. To find out which questions to answer, divide the last 2 digits of your UTEP ID by 3.

3.1 Prove that the smoothest line connecting two points is a straight line
3.2 What is machine learning: what do we know, and what do we want
3.3 What is the main objective of the traditional neural network?
3.4 Why in the traditional neural network, we only use linear and 1-input nonlinear neurons
3.5 Why cannot we have 1- or 2-layer neural networks?
3.6 Why we use L-NL-L networks and not NL-L-NL ones?
3.7 Explain why 3-layer neural networks are universal approximators
3.8 Which activation function is used in traditional neural networks? Why (biological answer).
3.9 What do we want to minimize when we train a neural network
3.10 What method do we use to minimize the corresponding objective function?
3.11 Explain why gradient method is used for optimization
3.12 Explain how gradient descent leads to backpropagation formulas for W0 and Wk.

Feel free to answer more questions, for extra credit.

4. (For extra credit) Provide a mathematical explanation of why sigmoid is used in traditional neural networks.