Given f(x)=x^2-x and g(x)=ax+b what are a and b if fog=gof .

We'll compose the functions f and
g.

fog(x) = f(g(x)) = [g(x)]^2 -
g(x)

f(g(x)) = (ax+b)^2 - ax -
b

gof(x) = g(f(x)) = a*f(x) +
b

g(f(x)) = a*(x^2 - x) + b

To
determine a and b, we'll impose the constraint given by
enunciation:

fog=gof

(ax+b)^2
- ax - b = a*(x^2 - x) + b

We'll expand the square from the
left side and we'll remove the brackets from the right
side:

a^2*x^2 + 2axb + b^2 - ax - b = ax^2 - ax +
b

We'll move all terms to one
side:

x^2(a^2 - a) + x(2ab - a + a) + b^2 - 2b =
0

Comparing, we'll get:

a^2 -
a = 0

a(a-1) = 0

a = 0 and a-1
= 0 => a = 1

2ab  = 0 => b =
0

b^2 - 2b = 0

b(b - 2) =
0

b = 0 and b =
2

The possible values are: a = {0; 1} and b =
{0 ; 2}.