Computational Intelligence for Engineering Solutions: Invariance-Based Approach
Homeworks for the course CS 5354/CS 4365, Fall 2022

Submit solutions via email to vladik@utep.edu.

1. (Due September 1) We say that a function f(x) is scale-shift-invariant if for every λ > 0, there exists a value y0 for which y = f(x) implies that y' = f(x'), where y' = y + y0 and x = λ * x.

Write down, in detail, the proof that if a differentiable function is scale-shift-invariant, then f(x) = A + a * ln(x) for some A and a.

Comment. Follow the pattern of proof about scale-scale invariance (in the handout) and shift-scale invariance (what we did in class). The main ideas of this proof are given in the Appendix to paper 6, you just need to describe it in all the detail.

Solution to Homework 1

2. (Due September 8) Suppose that you know the values of some quantity v in three points x1, x2, and x3, these values are v1 = 1, v2 = 2, and v3 = 3. Based on this information, you want to use the inverse distance weighting technique (as described in Paper 3 and as we described in class) to predict the value v of this quantity as a point x for which d(x1, x) = 10, d(x2, x) = 20, and d(x3, x) = 30. Take a = −1.

Solution to Homework 2

3. (Due September 8) Use calculus to find the value x for which the following function attains is minimum x2 + 2 * x + 1. What is the value of this minimum?

Solution to Homework 3

4. (Due September 13) Describe a function y = 1/(1 + exp(−x)) as a composition of invariant functions. Comment. This function is actively used in neural networks.

Solution to Homework 4

5. (Due September 13) Describe a function √(x2 + x4) as a scale-invariant combination of two scale-scale-invariant functions. Hint: use the fact that x4 = (x2)2.

Solution to Homework 5

6. (Due September 13) Prove that the family of all cubic polynomials

c0 + c1 * x + c2 * x2 + c3 * x3
is invariant with respect to shifts x → x + x0 and scalings x → λ * x, i.e., that if we substitute x' = x + x0 or x' = λ * x into this expression, we still get a cubic polynomial -- with different coefficients c'i.

Solution to Homework 6

7. (Due September 13 for extra credit, due September 20 for regular credit) As alternatives, let us consider families of the type {C*f(x)}C, where f(x) is fixed and C can take any value. Let us define scaling Tλ as an operation that transforms a family {C*f(x)}C into a new family {C*f(λ*x)}C. Prove that if an optimality criterion on the set of all such alternatives is final and scale-invariant, then each function which from the optimal family is a power law f(x) = A * xa. Hint: first, follow the general proof that we had in class about the function optimal with respect to a T-invariant criterion, and then use the result that we proved in class -- that every scale-scale-invariant function is described by the power law.

Second hint: Follow the logic of a similar proof that we had in class.

Solution to Homework 7

8. (Due September 15) Submit a report on the progress of your project.

9. (Due September 29) Use what we learned so far to explain the following two empirical formulas: y = x2 * e−kx and y = x0.3 * ln(x).

Solution to Homework 9

10. (Due September 29) Show that the class of all functions obtained from 1/(1 + k * exp(−x)) by fractional-linear transformations is shift-invariant.

Solution to Homework 10

11. (Due September 29) Show that for each a, the class of all functions obtained from 1/(1 + xa) by fractional-linear transformations is scale-invariant.

Solution to Homework 11

12. (Due November 10) Prepare handwritten "cheat sheet" for Test 2, with no more than 5 double-size pages, scan it, and send to the instructor. Students who got A on Test 2 do not need to do it.