Official solution
Let the wire is cut into two pieces of length x and 20 - x.
Area of square = \[{\left( {{x \over 4}} \right)^2}\]
Area of regular hexagon = \[6 \times {{\sqrt 3 } \over 4}{\left( {{{20 - x} \over 6}} \right)^2}\]
Total area = \[A(x) = {{{x^2}} \over {16}} + {{3\sqrt 3 } \over 2}{{{{(20 - x)}^2}} \over {36}}\]
\[A'(x) = {{2x} \over {16}} + {{3\sqrt 3 \times 2} \over {2 \times 36}}(20 - x)( - 1)\]
A'(x) = 0 at \[x = {{40\sqrt 3 } \over {3 + 2\sqrt 3 }}\]
Length of side of regular Hexagon \[ = {1 \over 6}(20 - x)\]
\[ = {1 \over 6}\left( {20 - {{4.\sqrt 3 } \over {3 + 2\sqrt 3 }}} \right)\]
\[ = {{10} \over {2 + 2\sqrt 3 }}\]