{"id":410,"date":"2019-12-04T09:38:22","date_gmt":"2019-12-04T09:38:22","guid":{"rendered":"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?page_id=410"},"modified":"2020-02-21T06:48:49","modified_gmt":"2020-02-21T06:48:49","slug":"neural-network-2","status":"publish","type":"page","link":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/?page_id=410","title":{"rendered":"Re\u021bele neurale"},"content":{"rendered":"<p><strong>COMPARA\u021aIE \u00ceNTRE METODE DE \u00ceNV\u0102\u021aARE DE TIP BACKPROPAGATION<\/strong><\/p>\n<p><strong><span style=\"text-decoration: underline;\">Abstract:<\/span><\/strong> Acest articol analizeaz\u0103 \u0219i compar\u0103 diverse \u00eembun\u0103t\u0103\u021biri aduse metodei backpropagation de ajustare a ponderilor unei re\u021bele feed-forward. Pentru aceasta se simuleaz\u0103 comportarea re\u021belelor pentru 6 aplica\u021bii tipice diverse. \u00cen plus, se propune \u0219i o metod\u0103 cu pas variabil \u0219i se demonstreaz\u0103 prin\u00a0 simulare superioritatea acesteia.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">1.Introducere<\/span><\/strong>Orice re\u021bea feed-forward are o structur\u0103 \u00een straturi. Fiecare strat este\u00a0<img loading=\"lazy\" class=\" wp-image alignright\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig1a.png\" alt=\"\" width=\"339\" height=\"221\" \/>compus din unit\u0103\u021bi care primesc intr\u0103rile de la unit\u0103\u021bile din stratul imediat anterior \u0219i \u00ee\u0219i trimit ie\u0219irea c\u0103tre unit\u0103\u021bile din stratul urm\u0103tor.<br \/>\nRegula de determinare a ponderilor sinaptice se nume\u0219te <strong>regula back-propagation<\/strong>. De\u0219i este posibil, nu este necesar s\u0103 aplic\u0103m aceast\u0103 regul\u0103 la mai mult de un strat de neuroni ascun\u0219i, pentru c\u0103 s-a ar\u0103tat <strong>[3]<\/strong> c\u0103 un singur strat de unit\u0103\u021bi ascunse este suficient pentru a aproxima cu o anumit\u0103 precizie orice func\u021bie cu un num\u0103r finit de discontinuit\u0103\u021bi, dac\u0103 unit\u0103\u021bile ascunse au fost activate de func\u021bii neliniare. De cele mai multe ori, \u00een aplica\u021bii se folose\u0219te o re\u021bea feed-forward cu un singur strat de unit\u0103\u021bi ascunse \u0219i cu func\u021bia sigmoid\u0103 de activare pentru unit\u0103\u021bi.<br \/>\nPentru o re\u021bea generic\u0103 cu trei straturi (<strong>figura 1.a.<\/strong>) starea re\u021belei este dat\u0103 de ecua\u021biile:<\/p>\n<p><img title=\"y_{m}=\\mathfrak{F}(a_{_m});\\textrm{&amp;space;}a_{_m}=\\sum_{j=1}^{J}q_{mj}\\cdot v_{j}-s_{m},\\textrm{&amp;space;}m=\\overline{1,M}\\textrm{&amp;space;}(1)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?y_{m}=\\mathfrak{F}(a_{_m});\\textrm{&amp;space;}a_{_m}=\\sum_{j=1}^{J}q_{mj}\\cdot&amp;space;v_{j}-s_{m},\\textrm{&amp;space;}m=\\overline{1,M}\\textrm{&amp;space;}(1)\" alt=\"\" \/><\/p>\n<p><img title=\"v_{j}=\\mathfrak{F}(\\overline{a_{j}});\\textrm{ }a_{j}=\\sum_{i=1}^{I}w_{ji}\\cdot x_{i}-c_{j},\\textrm{ }j=\\overline{1,J}\\textrm{ }(2)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?v_{j}=\\mathfrak{F}(\\overline{a_{j}});\\textrm{&amp;space;}a_{j}=\\sum_{i=1}^{I}w_{ji}\\cdot&amp;space;x_{i}-c_{j},\\textrm{&amp;space;}\\textrm{&amp;space;}j=\\overline{1,J}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}(2)\" alt=\"\" \/><\/p>\n<p>Ponderile sinaptice (w<sub>ji<\/sub> \u0219i q<sub>mj<\/sub> ) \u0219i poten\u021bialele de prag ( s<sub>m<\/sub> \u0219i c<sub>j<\/sub>) trebuie astfel alese \u00eenc\u00e2t eroarea total\u0103:<\/p>\n<p><img title=\"\\mathbf{J_{s}}(\\mathbf{ W,Q,s,c})=\\sum_{\\mu }^{ } J__{s}^{\\mu }\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\mathbf{J_{s}}(\\mathbf{&amp;space;W,Q,s,c})=\\sum_{\\mu&amp;space;}^{&amp;space;}&amp;space; \\mathbf{J_{s}^{\\mu&amp;space;}}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}\\textrm{&amp;space;}(3)\" alt=\"\" \/><\/p>\n<p>s\u0103 fie c\u00e2t mai mic\u0103.\u00a0<img title=\"J_{s}^{\\mu }\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;J_{s}^{\\mu&amp;space;}\" alt=\"\" \/> este eroarea p\u0103tratic\u0103 la ie\u0219ire pentru patternul \u03bc:<br \/>\n<img title=\"\\mathbf{J_{s}^{\\mu}}=\\frac{1}{2}\\cdot \\sum_{m=1}^{M}\\left [ d_{m}^{\\mu }-\\mathfrak{F(a_{m}^{\\mu })} \\right ]^{2} (4)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\mathbf{J_{s}^{\\mu}}=\\frac{1}{2}\\cdot&amp;space;\\sum_{m=1}^{M}\\left&amp;space;[&amp;space;d_{m}^{\\mu&amp;space;}-\\mathfrak{F(a_{m}^{\\mu&amp;space;})}&amp;space;\\right&amp;space;]^{2}\\textrm{&amp;space;}(4)\" alt=\"\" \/><\/p>\n<p>unde d<sup>\u03bc<\/sup> este vectorul ie\u0219ire dorit pentru clasa\u00a0\u03bc . Se observ\u0103 deci c\u0103 regula back-propagation este o generalizare a regulii delta. \u00cen consecin\u021b\u0103, trebuie s\u0103 calcul\u0103m gradientul lui J, \u00een raport cu fiecare parametru \u0219i apoi s\u0103 modific\u0103m propor\u021bional cu acest gradient valoarea respectivului parametru. \u00cen prima faz\u0103 consider\u0103m numai conexiunile sinaptice ale neuronilor de ie\u0219ire:<br \/>\n<img loading=\"lazy\" class=\"alignnone\" title=\" \\Delta q_{mj}= -\\rho \\cdot \\frac{\\partial J_{s}}{\\partial q_{mj}}=-\\rho \\cdot \\sum_{\\mu}^{ }\\frac{\\partial J_{s}^{\\mu }}{\\partial q_{mj}}\\cdot \\frac{\\partial a_{m}^{\\mu }}{\\partial q_{mj}} =\\rho \\cdot \\sum_{\\mu}^{ }[d_{m}^{\\mu }-\\mathfrak{F(a_{m}^{\\mu })}]\\cdot \\mathfrak{F(a_{m}^{\\mu })}\\cdot \\frac{\\partial a_{m}^{\\mu }}{\\partial q_{mj}}\\equiv \\rho \\cdot \\sum_{\\mu}^{ }\\Delta _{m}^{\\mu \\cdot V_{j}^{\\mu }} (5))\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;q_{mj}=&amp;space;-\\rho&amp;space;\\cdot&amp;space;\\frac{\\partial&amp;space;J_{s}}{\\partial&amp;space;q_{mj}}=-\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu}^{&amp;space;}\\frac{\\partial&amp;space;J_{s}^{\\mu&amp;space;}}{\\partial&amp;space;q_{mj}}\\cdot&amp;space;\\frac{\\partial&amp;space;a_{m}^{\\mu&amp;space;}}{\\partial&amp;space;q_{mj}}&amp;space;=\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu}^{&amp;space;}[d_{m}^{\\mu&amp;space;}-\\mathfrak{F(a_{m}^{\\mu&amp;space;})}]\\cdot&amp;space;\\mathfrak{F(a_{m}^{\\mu&amp;space;})}\\cdot&amp;space;\\frac{\\partial&amp;space;a_{m}^{\\mu&amp;space;}}{\\partial&amp;space;q_{mj}}\\equiv&amp;space;\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu}^{&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;\\cdot&amp;space;}V_{j}^{\\mu&amp;space;}&amp;space;(5)\" alt=\"\" width=\"593\" height=\"121\" \/><\/p>\n<p><img loading=\"lazy\" class=\"alignnone\" title=\" \\Delta S_{m}= -\\rho \\cdot \\frac{\\partial J_{s}}{\\partial J_{m}}=\\rho \\cdot \\sum_{\\mu}^{ }[d_{m}^{\\mu }-\\mathfrak{F(a_{m}^{\\mu })}]\\cdot \\mathfrak{F(a_{m}^{\\mu })}\\cdot \\frac{\\partial a_{m}^{\\mu }}{\\partial S_{m}}=- \\rho \\cdot \\sum_{\\mu}^{ }\\Delta _{m}^{\\mu }= \\rho \\cdot \\sum_{\\mu}^{ }\\Delta _{m}^{\\mu }\\cdot (-1)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;S_{m}=&amp;space;-\\rho&amp;space;\\cdot&amp;space;\\frac{\\partial&amp;space;J_{s}}{\\partial&amp;space;J_{m}}=\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu}^{&amp;space;}[d_{m}^{\\mu&amp;space;}-\\mathfrak{F(a_{m}^{\\mu&amp;space;})}]\\cdot&amp;space;\\mathfrak{F(a_{m}^{\\mu&amp;space;})}\\cdot&amp;space;\\frac{\\partial&amp;space;a_{m}^{\\mu&amp;space;}}{\\partial&amp;space;s_{m}}=-&amp;space;\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu}^{&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;}=&amp;space;\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu}^{&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;}\\cdot&amp;space;(-1) (6)\" alt=\"\" width=\"592\" height=\"121\" \/><\/p>\n<p><img title=\"\\small \\Delta _{m}^{\\mu }=[d_{m}^{\\mu }-\\mathfrak{F(a_{m}^{\\mu })}]\\cdot \\mathfrak{F(a_{m}^{\\mu })} (7)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\small&amp;space;\\Delta&amp;space;_{m}^{\\mu&amp;space;}=[d_{m}^{\\mu&amp;space;}-\\mathfrak{F(a_{m}^{\\mu&amp;space;})}]\\cdot&amp;space;\\mathfrak{F(a_{m}^{\\mu&amp;space;})}&amp;space;(7)\" alt=\"\" \/><\/p>\n<p>\u00cen pasul urm\u0103tor consider\u0103m parametrii asocia\u021bi cu conexiunile sinaptice dintre stratul de intrare \u0219i cel ascuns. Procedura este asem\u0103n\u0103toare, numai c\u0103 trebuie f\u0103cute dou\u0103 substitu\u021bii:<\/p>\n<p><img title=\"\\Delta w_{ji}=-\\rho \\cdot \\frac{\\partial J_{s}}{\\partial w_{ji}}=-\\rho \\cdot \\sum_{\\mu }^{ }\\sum_{m=1}^{M}\\frac{\\partial J_{s}^{\\mu }}{\\partial a_{m}^{\\mu }}\\cdot \\frac{{\\partial a_{m}^{\\mu }}}{\\partial v_{j}}\\cdot \\frac{\\partial v_{j}}{\\partial w_{ji}}=\\rho \\cdot \\sum_{\\mu }^{ }\\sum_{m=1}^{M}[d_{m}^{\\mu }-\\mathfrak{F(a_{m}^{\\mu })}]\\cdot \\mathfrak{F(a_{m}^{\\mu })}\\cdot\\frac{{\\partial a_{m}^{\\mu }}}{\\partial v_{j}}\\cdot \\frac{\\partial v_{j}}{\\partial w_{ji}}=\\rho \\cdot \\sum_{\\mu }^{ }\\sum_{m=1}^{M}\\Delta _{m}^{\\mu }\\cdot q_{mj}\\cdot \\mathfrak{F}(\\overline{a_{J}^{\\mu }})\\cdot \\frac{\\partial\\overline{ a_{J}}}{\\partial w_{ji}}\\equiv \\rho \\cdot \\sum_{\\mu }^{ } \\overline{\\Delta _{J}^{\\mu }}\\cdot x_{i}^{\\mu } (8)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;w_{ji}=-\\rho&amp;space;\\cdot&amp;space;\\frac{\\partial&amp;space;J_{s}}{\\partial&amp;space;w_{ji}}=-\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu&amp;space;}^{&amp;space;}\\sum_{m=1}^{M}\\frac{\\partial&amp;space;J_{s}^{\\mu&amp;space;}}{\\partial&amp;space;a_{m}^{\\mu&amp;space;}}\\cdot&amp;space;\\frac{{\\partial&amp;space;a_{m}^{\\mu&amp;space;}}}{\\partial&amp;space;v_{j}}\\cdot&amp;space;\\frac{\\partial&amp;space;v_{j}}{\\partial&amp;space;w_{ji}}=\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu&amp;space;}^{&amp;space;}\\sum_{m=1}^{M}[d_{m}^{\\mu&amp;space;}-\\mathfrak{F(a_{m}^{\\mu&amp;space;})}]\\cdot&amp;space;\\mathfrak{F(a_{m}^{\\mu&amp;space;})}\\cdot\\frac{{\\partial&amp;space;a_{m}^{\\mu&amp;space;}}}{\\partial&amp;space;v_{j}}\\cdot&amp;space;\\frac{\\partial&amp;space;v_{j}}{\\partial&amp;space;w_{ji}}=\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu&amp;space;}^{&amp;space;}\\sum_{m=1}^{M}\\Delta&amp;space;_{m}^{\\mu&amp;space;}\\cdot&amp;space;q_{mj}\\cdot&amp;space;\\mathfrak{F}(\\overline{a_{J}^{\\mu&amp;space;}})\\cdot&amp;space;\\frac{\\partial\\overline{&amp;space;a_{J}}}{\\partial&amp;space;w_{ji}}\\equiv&amp;space;\\rho&amp;space;\\cdot&amp;space;\\sum_{\\mu&amp;space;}^{&amp;space;}&amp;space;\\overline{\\Delta&amp;space;_{J}^{\\mu&amp;space;}}\\cdot&amp;space;x_{i}^{\\mu&amp;space;}&amp;space;(8)\" alt=\"\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img title=\"\\Delta c_{j}=-\\rho \\cdot \\frac{\\partial J_{s} }{\\partial c_{j}}\\equiv -\\rho\\cdot \\sum_{\\mu }^{ }\\overline{\\Delta _{J}^{\\mu }}= \\rho\\cdot \\sum_{\\mu }^{ }\\overline{\\Delta _{J}^{\\mu }}\\cdot (-1) (9)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;c_{j}=-\\rho&amp;space;\\cdot&amp;space;\\frac{\\partial&amp;space;J_{s}&amp;space;}{\\partial&amp;space;c_{j}}\\equiv&amp;space;-\\rho\\cdot&amp;space;\\sum_{\\mu&amp;space;}^{&amp;space;}\\overline{\\Delta&amp;space;_{J}^{\\mu&amp;space;}}=&amp;space;\\rho\\cdot&amp;space;\\sum_{\\mu&amp;space;}^{&amp;space;}\\overline{\\Delta&amp;space;_{J}^{\\mu&amp;space;}}\\cdot&amp;space;(-1)&amp;space;(9)\" alt=\"\" \/><\/p>\n<p>unde:<\/p>\n<p><img title=\"\\overline{\\Delta _{J}^{\\mu }}=\\left [ \\sum_{m}^{ }\\Delta _{m}^{\\mu }\\cdot q_{mj} \\right ]\\cdot \\mathfrak{F(\\overline{a_{j}^{\\mu }})}(10)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\overline{\\Delta&amp;space;_{J}^{\\mu&amp;space;}}=\\left&amp;space;[&amp;space;\\sum_{m}^{&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;}\\cdot&amp;space;q_{mj}&amp;space;\\right&amp;space;]\\cdot&amp;space;\\mathfrak{F(\\overline{a_{j}^{\\mu&amp;space;}})}(10)\" alt=\"\" \/><\/p>\n<p>De observat c\u0103 ecua\u021biile de ajustare a ponderilor sinaptice <strong>(8) <\/strong>\u0219i <strong>(9) <\/strong>au aceea\u0219i form\u0103 cu ecua\u021biile sinaptice <strong>(5)<\/strong> \u0219i<strong> (6)<\/strong>, numai expresia lui <img title=\"\\overline{\\Delta _{J}^{\\mu }}\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\overline{\\Delta&amp;space;_{J}^{\\mu&amp;space;}}\" alt=\"\" \/> difer\u0103 de\u00a0<img title=\"\\Delta _{m}^{\\mu }\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;_{m}^{\\mu&amp;space;}\" alt=\"\" \/> , din care se poate ob\u021bine recursiv.<br \/>\nMetoda backpropagation a fost extins\u0103 \u0219i la re\u021belele recurente <strong>[11, 12, 13], <\/strong>care din acel moment au devenit tot mai populare.<br \/>\nSchema de corec\u021bie a erorilor lucreaz\u0103 ca \u0219i cum informa\u021bia despre abaterea de la ie\u0219irea dorit\u0103 s-ar propaga \u00eenapoi (backward) prin re\u021bea, \u201dcontra curentului\u201d conexiunilor sinaptice. Este \u00eendoielnic, cu toate c\u0103 nu complet imposibil, ca o asemenea procedur\u0103 s\u0103 poat\u0103 fi realizat\u0103 de re\u021belele neurale biologice. Ceea ce este sigur, este c\u0103 algoritmul propag\u0103rii \u00eenapoi a erorii este foarte potrivit pentru calculatoarele electronice, at\u00e2t \u00een implement\u0103rile hardware, c\u00e2t \u0219i software. Mai recent, s-a produs \u0219i o regul\u0103 de tip \u201dbackpropagation\u201d care se bazeaz\u0103 nu pe minimizarea erorii p\u0103tratice medii totale, ci pe maximizarea informa\u021biei K\u00fcllbach <strong>[2]<\/strong>.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" class=\"aligncenter wp-image size-large\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/figura1b.png\" alt=\"\" width=\"640\" height=\"287\" \/><\/p>\n<p>Arhitectura din <strong>figura 1.a.<\/strong> ca \u0219i rela\u021biile <strong>(5)-(10) <\/strong>pot fi generalizate la o re\u021bea feedforward cu R straturi (R-2 straturi ascunse) ob\u021bin\u00e2nd arhitectura din <strong>figura1.b.<\/strong> Criteriul de eroare care trebuie minimizat este tot eroarea medie p\u0103tratic\u0103 determinat\u0103 pe mul\u021bimea tuturor exemplelor de antrenament:<br \/>\n<img title=\"J=\\sum_{\\mu }{ }\\sum_{j_{1}}^{H_{1}}(d_{j_{1}}^{\\mu }-v_{j_{1}}^{\\mu })^{2} (11)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;J=\\sum_{\\mu&amp;space;}{&amp;space;}\\sum_{j_{1}}^{H_{1}}(d_{j_{1}}^{\\mu&amp;space;}-v_{j_{1}}^{\\mu&amp;space;})^{2}&amp;space;(11)\" alt=\"\" \/><\/p>\n<p>Ie\u0219irea re\u021belei va fi tocmai ie\u0219irea ultimului strat neuronal:<\/p>\n<p>&nbsp;<\/p>\n<p><img title=\"v _{j_{c}}^{\\mu }=\\mathfrak{F}\\left ( \\sum_{j_{c+1}=1}^{H_{c+1}} w_{j_{c},j_{c+1}}\\cdot v _{j_{c+1}}^{\\mu }-s_{j_{c}} \\right )(12)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;v&amp;space;_{j_{c}}^{\\mu&amp;space;}=\\mathfrak{F}\\left&amp;space;(&amp;space;\\sum_{j_{c+1}=1}^{H_{c+1}}&amp;space;w_{j_{c},j_{c+1}}\\cdot&amp;space;v&amp;space;_{j_{c+1}}^{\\mu&amp;space;}-s_{j_{c}}&amp;space;\\right&amp;space;)(12)\" alt=\"\" \/><\/p>\n<p>Aplic\u00e2nd regula backpropagation se ob\u021bin urm\u0103toarele rela\u021bii:<\/p>\n<p>&nbsp;<\/p>\n<p><img title=\"\\Delta w_{j_{c},j_{c+1}}=\\sum_{\\mu }^{ }\\Delta_{j_{c}}^{\\mu } \\cdot v _{j_{c+1}}^{\\mu }(13)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;w_{j_{c},j_{c+1}}=\\sum_{\\mu&amp;space;}^{&amp;space;}\\Delta_{j_{c}}^{\\mu&amp;space;}&amp;space;\\cdot&amp;space;v&amp;space;_{j_{c+1}}^{\\mu&amp;space;}(13)\" alt=\"\" \/><\/p>\n<p><img title=\"\\Delta _{j_{c}}^{\\mu }=\\left ( \\sum_{j_{c-1}=1}^{H_{c-1}}\\Delta _{j_{c-1}}\\cdot w_{j_{c-1}j_{c}}\\right )\\cdot \\mathfrak{F}\\left ( v_{j_{c}}^{\\mu } \\right )(14)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;_{j_{c}}^{\\mu&amp;space;}=\\left&amp;space;(&amp;space;\\sum_{j_{c-1}=1}^{H_{c-1}}\\Delta&amp;space;_{j_{c-1}}\\cdot&amp;space;w_{j_{c-1}j_{c}}\\right&amp;space;)\\cdot&amp;space;\\mathfrak{F}\\left&amp;space;(&amp;space;v_{j_{c}}^{\\mu&amp;space;}&amp;space;\\right&amp;space;)(14)\" alt=\"\" \/><\/p>\n<p><img title=\"\\Delta _{j_{1}}^{\\mu }= \\sum_{j_{1}=1}^{H_{1}} \\left ( d_{j_{1}}^{\\mu }-v_{j_{1}}^{\\mu } \\right )\\cdot \\mathfrak{F}\\left ( v_{j_{1}}^{\\mu } \\right )(15)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;_{j_{1}}^{\\mu&amp;space;}=&amp;space;\\sum_{j_{1}=1}^{H_{1}}&amp;space;\\left&amp;space;(&amp;space;d_{j_{1}}^{\\mu&amp;space;}-v_{j_{1}}^{\\mu&amp;space;}&amp;space;\\right&amp;space;)\\cdot&amp;space;\\mathfrak{F}\\left&amp;space;(&amp;space;v_{j_{1}}^{\\mu&amp;space;}&amp;space;\\right&amp;space;)(15)\" alt=\"\" \/><\/p>\n<p><img title=\"\\Delta S_{j_{c}}=\\sum_{\\mu }^{ }\\Delta_{j_{c}}^{\\mu } \\cdot (-1)(16)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\Delta&amp;space;S_{j_{c}}=\\sum_{\\mu&amp;space;}^{&amp;space;}\\Delta_{j_{c}}^{\\mu&amp;space;}&amp;space;\\cdot&amp;space;(-1)(16)\" alt=\"\" \/><\/p>\n<p>Metoda backpropagation fiind o metod\u0103 de minimizare a erorii medii p\u0103tratice pe \u00eentreaga mul\u021bime a exemplelor de antrenament, metod\u0103 bazat\u0103 pe gradientul negativ, sufer\u0103 de deficien\u021bele generale ale tehnicilor de gradient. Aceste deficien\u021be sunt, \u00een general, dou\u0103:<\/p>\n<ul>\n<li>Rata de \u00eenv\u0103\u021bare, \u03c1, care d\u0103 m\u0103rimea pasului pe direc\u021bia gradientului trebuie s\u0103 fie suficient de mic\u0103. Aceasta deoarece gradientul este o m\u0103sur\u0103 local\u0103, iar dac\u0103 \u00a0este prea mare s-ar putea ca m\u0103sura J a erorii s\u0103 nu scad\u0103, ci doar s\u0103 oscileze, vectorul coeficien\u021bilor sinaptici s\u0103rind de pe o parte pe cealalt\u0103 a \u201dgropii\u201d \u00een care se g\u0103se\u0219te minimul local. Pe de alt\u0103 parte, un \u03c1\u00a0prea mic conduce la o vitez\u0103 prea mic\u0103 de convergen\u021b\u0103, ceea ce conduce la o \u00eenv\u0103\u021bare prea lent\u0103. Chiar dac\u0103 se poate determina (cu mare consum de timp) un \u03c1\u00a0optim, acest optim se modific\u0103 pe m\u0103sur\u0103 ce sistemul \u00eenva\u021b\u0103; de asemenea el difer\u0103 de la problem\u0103 la problem\u0103.<\/li>\n<li> Indiferent de c\u00e2t de rapid atinge minimul, acesta va fi minimul <strong>local<\/strong> cel mai apropiat \u0219i din \u00a0acesta nu va putea ie\u0219i pentru c\u0103 tehnica backpropagation, fiind o metod\u0103 de gradient, este o metod\u0103 ce exploateaz\u0103 propriet\u0103\u021bile locale ale func\u021biei criteriu. \u00cen plus, se poate deduce: cu c\u00e2t dimensiunea spa\u021biului de c\u0103utare (spa\u021biul ponderilor) este mai mare (deci avem un num\u0103r mai mare de neuroni \u00een straturile ascunse), cu at\u00e2t cre\u0219te num\u0103rul minimelor locale \u0219i deci \u0219i \u201d\u0219ansa\u201d de a e\u0219ua \u00eentr-unul dintre acestea.<\/li>\n<\/ul>\n<p>Cu aceste perspective vom compara metoda backpropagation clasic\u0103 cu unele dintre \u00eembun\u0103t\u0103\u021birile prezentate \u00een literatur\u0103 \u0219i, \u00een final, vom propune o metod\u0103 care s\u0103 convearg\u0103 mai rapid \u0219i s\u0103 poat\u0103 evada din minimele locale.<\/p>\n<p>Pentru aceasta vom simula comportarea re\u021belelor respective \u00een rezolvarea a 6 probleme tipice.<\/p>\n<p><strong><span style=\"text-decoration: underline;\">2.Problemele la a c\u0103ror rezolvare s-au testat metodele <\/span><\/strong><\/p>\n<p>Pentru fiecare problem\u0103 se va specifica num\u0103rul de unit\u0103\u021bi de intrare, de ie\u0219ire \u0219i din stratul ascuns. Pentru determinarea num\u0103rului de unit\u0103\u021bi din acest strat se va \u021bine cont de rezultatul altor simul\u0103ri <strong>[17]<\/strong> unde se stabile\u0219te empiric faptul c\u0103, pentru a putea rezolva problemele de clasificare este bine ca acest num\u0103r s\u0103 fie aproximativ egal cu media aritmetic\u0103 dintre num\u0103rul neuronilor de intrare \u0219i num\u0103rul celor de ie\u0219ire, dar nu mai mic dec\u00e2t 10.<strong> <\/strong><br \/>\n2.1.\u00a0 <span style=\"text-decoration: underline;\">BINAR<\/span><\/p>\n<p>Aceast\u0103 problem\u0103 const\u0103 \u00een determinarea parit\u0103\u021bii unui cuv\u00e2nt de 8 bi\u021bi (EXCLUSIVE-OR Task) <strong>[6,9]<\/strong>.<\/p>\n<p>Re\u021belele folosite vor avea deci 8 unit\u0103\u021bi de intrare \u0219i una de ie\u0219ire care va lua valoarea 0 pentru un cuv\u00e2nt cu un num\u0103r par de bi\u021bi de 1 \u0219i valoarea 1 \u00een caz contrar. Num\u0103rul de unit\u0103\u021bi ascunse este egal cu 10.<br \/>\n\u00cen aceast\u0103 problem\u0103 am ales antrenarea re\u021belei pe mul\u021bimea tuturor exemplelor posibile, deci pe 256 de exemple.<\/p>\n<p>2.2.\u00a0 <span style=\"text-decoration: underline;\">COUNTER<\/span><\/p>\n<p>Se propune realizarea unui num\u0103r\u0103tor de bi\u021bi de 1 dintr-un cuv\u00e2nt binar de 4 bi\u021bi <strong>[6,9].<\/strong> Deci, vom avea 4 neuroni de intrare \u0219i 5 de ie\u0219ire, vectorul ie\u0219irilor apar\u021bin\u00e2nd mul\u021bimii {(1,0,0,0,0)<sup>T<\/sup>,(0,1,0,0,0)<sup>T<\/sup>,\u00a0(0,0,1,0,0)<sup>T<\/sup> (0,0,0,1,0)<sup>T<\/sup> (0,0,0,0,1)<sup>T<\/sup>} fiind egal cu vectorul i (i=1, &#8230; , 5) dac\u0103 vectorul de intrare are i-1 bi\u021bi de 1. Num\u0103rul ales pentru unit\u0103\u021bile ascunse este 10. Am antrenat re\u021belele pe toate exemplele de antrenament posibile; a\u0219adar, num\u0103rul acestora este 2<sup>4<\/sup> = 16.<\/p>\n<p>2.3.\u00a0 <span style=\"text-decoration: underline;\">MULTIPLEXOR<\/span><\/p>\n<p>Re\u021beaua neural\u0103 care \u00eel va simula trebuie s\u0103 \u00eenve\u021be ca vectorului de intrare de 3 bi\u021bi (3 neuroni de intrare) (b<sub>1<\/sub>,b<sub>2<\/sub>,b<sub>3<\/sub>)<sup>T<\/sup> s\u0103-i ata\u0219eze la ie\u0219ire vectorul de 8 bi\u021bi (0,0,..,0,1,0,&#8230;,0)<sup>T<\/sup> cu un singur 1 \u00een pozi\u021bia i =b<sub>1<\/sub> \u20222<sup>2<\/sup> +b<sub>2\u2022<\/sub> 2 + b<sub>3<\/sub>. Num\u0103rul exemplelor de antrenament este 2<sup>3<\/sup>=8 <strong>[6],<\/strong> iar num\u0103rul unit\u0103\u021bilor ascunse este, evident, tot egal cu 10<\/p>\n<p>2.4.\u00a0 <span style=\"text-decoration: underline;\">5x<\/span><span style=\"text-decoration: underline;\">5\u00a0 TABLE<\/span><\/p>\n<p>Aceast\u0103 problem\u0103 este problema recunoa\u0219terii (clasific\u0103rii) liniilor \u0219i coloanelor \u00eentr-o figur\u0103 binar\u0103. Imaginea este reprezentat\u0103 de o re\u021bea de 5&#215;5 (5 linii \u0219i 5 coloane, adic\u0103 25 de celule de intrare, respectiv 25 de neuroni de intrare). Deoarece fiecare imagine poate con\u021bine at\u00e2t linii c\u00e2t \u0219i coloane vom avea 2 neuroni de ie\u0219ire; num\u0103rul neuronilor din stratul ascuns l-am ales egal cu 14.<br \/>\nConsider\u00e2nd imaginea dat\u0103 de matricea C<sub>[n,m]<\/sub> , (n, m, = 1, &#8230; , 5), atunci vectorii de intrare <strong>x<\/strong>[i] se vor forma lu\u00e2nd i = 3\u00b7(n-1)+m. Am ales simularea \u00eenv\u0103\u021b\u0103rii pentru o mul\u021bime de 16 exemple, pentru fiecare d\u00e2nd ca \u00een orice \u00eenv\u0103\u021bare supervizat\u0103 \u0219i vectorul de ie\u0219ire dorit <strong>d: <\/strong><\/p>\n<p style=\"text-align: center;\">x\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 d<br \/>\n<span style=\"font-family: 'times new roman', times, serif;\">11111.00000.00000.00000.00000\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00000.11111.00000.00000.00000\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00000.00000.11111.00000.00000\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00000.00000.00000.11111.00000\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00000.00000.00000.00000.11111\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 10000.10000.10000.10000.10000\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 01000.01000.01000.01000.01000\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00100.00100.00100.00100.00100\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00010.00010.00010.00010.00010 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00001.00001.00001.00001.00001\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 11111.11111.00000.00000.00000\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00000.00000.00000.11111.11111\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a01 0<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 11000.11000.11000.11000.11000\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00011.00011.00011.00011.00011\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a00 1<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 10000.01000.00100.00010.00001\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01\/2 1\/2<\/span><br \/>\n<span style=\"font-family: 'times new roman', times, serif;\"> 00001.00010.00100.01000.10000\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a01\/2 1\/2<\/span><\/p>\n<p>2.5. <span style=\"text-decoration: underline;\">ASSOCIATIVE MEMORY<\/span><\/p>\n<p>Se testeaz\u0103 capacitatea de \u201dfeature extraction\u201d a re\u021belelor neurale, adic\u0103 posibilitatea de a re\u021bine \u201dformele\u201d de intrare intr-un vector de dimensiune mic\u0103. Prin urmare, re\u021beaua e \u00eenv\u0103\u021bat\u0103 ca unui vector de intrare de 16 bi\u021bi s\u0103-i corespund\u0103 acela\u0219i vector (tot de 16 bi\u021bi) la ie\u0219ire <strong>[9,10].<\/strong> Dificultatea const\u0103 \u00een aceea c\u0103 stratul ascuns are numai 4 neuroni. Am antrenat re\u021belele pe o mul\u021bime de 16 exemple: vectorii cu a i-a component\u0103 egal\u0103 cu 1 (i = 1, &#8230; , 16), iar celelalte componente egale cu 0.<\/p>\n<p>2.6.\u00a0 <span style=\"text-decoration: underline;\">FUNCTION<\/span><\/p>\n<p>Pe l\u00e2ng\u0103 re\u021belele recurente (capabile s\u0103 reprezinte serii de timp) se folosesc la emularea proceselor \u0219i re\u021bele feedforward, conectate \u00een arhitectura din <strong>figura 2.a. [14, 15].<\/strong><\/p>\n<p><img loading=\"lazy\" class=\"alignnone wp-image size-full\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/figura2a.png\" alt=\"\" width=\"1005\" height=\"593\" \/><\/p>\n<p>Fiind da\u021bi p \u0219i q se poate emula un proces cu o re\u021bea neural\u0103 cu m=p+q+1 intr\u0103ri \u0219i o ie\u0219ire. Not\u00e2nd reprezentarea realizat\u0103 de emulator prin \u03c6<sub>E<\/sub>( \u2022) \u0219i ie\u0219irea lui prin y&#8217; avem:<\/p>\n<p><img title=\"y'=\\varphi_{E} (X_{E})(17)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;y'=\\varphi_{E}&amp;space;(X_{E})(17)\" alt=\"\" \/><\/p>\n<p>unde:<\/p>\n<p><img title=\"x_{E}(k)=[y(k),y(k-1),...,y(k-p+1),u(k),u(k-1),...,u(k-q)]^{T}(18)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;x_{E}(k)=[y(k),y(k-1),...,y(k-p+1),u(k),u(k-1),...,u(k-q)]^{T}(18)\" alt=\"\" \/><\/p>\n<p>Emulatorul este antrenat s\u0103 minimizeze m\u0103sura erorii de emulare: y(k+1)-y&#8217;. \u00cen <strong>figura 2.a., <\/strong>prin z<sup>-1<\/sup> se \u00een\u021belege operatorul de \u00eent\u00e2rziere.<br \/>\nPentru testarea metodelor de \u00eenv\u0103\u021bare am ales un sistem de ordinul doi profund neliniar, care a fost folosit \u00een <strong>[5] <\/strong>\u0219i care \u00eel preia din <strong>[8]<\/strong>. Sistemul este descris de ecua\u021bia:<\/p>\n<p><img title=\"y(k)=\\frac{y(k-1)\\cdot y(k-2)\\cdot [y(k-1)+2,5]}{1+y^{2}(k-1)+y^{2}(k-2)}+u(k)(19)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;y(k)=\\frac{y(k-1)\\cdot&amp;space;y(k-2)\\cdot&amp;space;[y(k-1)+2,5]}{1+y^{2}(k-1)+y^{2}(k-2)}+u(k)(19)\" alt=\"\" \/><\/p>\n<p>unde u(k) este un semnal sinusoidal:<\/p>\n<p><img title=\"u(k)=sin(0,08\\cdot \\pi \\cdot k)(20)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;u(k)=sin(0,08\\cdot&amp;space;\\pi&amp;space;\\cdot&amp;space;k)(20)\" alt=\"\" \/><\/p>\n<p>Am generat 80 de perechi (u(k), y(k)), pornind de la y(-1)=0 \u0219i y(0)=1 \u0219i le-am folosit pentru antrenarea <strong>off-line<\/strong> a re\u021belelor.<br \/>\nA\u0219a cum se procedeaz\u0103 \u00een general, am presupus cunoscut ordinul procesului SISO (Single Input Single Output) \u0219i, de aceea, am ales o re\u021bea cu 3 unit\u0103\u021bi de intrare, o unitate de ie\u0219ire \u0219i 10 unit\u0103\u021bi ascunse. Unit\u0103\u021bile de intrare se conecteaz\u0103 ca \u00een <strong>figura 2.b. <\/strong>\u0219i anume, la momentul k, cele trei unit\u0103\u021bi x vor fi legate c\u00e2te una la y(k-2), y(k-1) \u0219i u(k), \u00een timp ce ie\u0219irea dorit\u0103, d, va fi considerat\u0103 y(k).<\/p>\n<p><img loading=\"lazy\" class=\"alignnone wp-image size-full\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/figura2b.png\" alt=\"\" width=\"891\" height=\"500\" \/><\/p>\n<p>De fapt, deoarece ie\u0219irea unui neuron este cuprins\u0103 \u00eentotdeauna \u00eentre 0 \u0219i 1, trebuie efectuat\u0103 o conversie a valorii lui y \u00een acest interval. Am ales o transformare liniar\u0103 a domeniului \u00een care ia valori y \u00een intervalul (0.15, 0.85) pentru a permite adaptarea re\u021belei \u0219i la semnale care ulterior perioadei de \u00eenv\u0103\u021bare ar ie\u0219i din plaja de valori de antrenament \u0219i, apoi, pe MaxDy care este modulul abaterii valorilor de la medie, de unde rezult\u0103:<\/p>\n<p><img title=\"d(k)=\\frac{1}{2}+0,7\\cdot \\frac{|y(k)-Medy|}{MaxDy}(21)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;d(k)=\\frac{1}{2}+0,7\\cdot&amp;space;\\frac{|y(k)-Medy|}{MaxDy}(21)\" alt=\"\" \/><\/p>\n<p>Spre deosebire de al\u021bi autori, nu efectuez o opera\u021bie asem\u0103n\u0103toare \u0219i pentru intr\u0103ri: o asemenea opera\u021bie ar u\u0219ura poate adaptarea, dar ar complica arhitectura pentru antrenament c\u0103ci ar trebui s\u0103 se fac\u0103 o distinc\u021bie \u00eentre tipurile de intr\u0103ri (care provin din intr\u0103rile procesului \u0219i care din ie\u0219irile acestuia).<\/p>\n<p>3. <span style=\"text-decoration: underline;\"><strong>Metodele de \u00eenv\u0103\u021bare<\/strong><\/span><br \/>\n3.1. <span style=\"text-decoration: underline;\">BackPropagation Classic<strong> (BP)<\/strong><\/span><br \/>\nLa metoda clasic\u0103 r\u0103m\u00e2ne la alegerea utilizatorului precizarea valorii ratei de \u00eenv\u0103\u021bare, adic\u0103 a m\u0103rimii pasului \u00een direc\u021bia gradientului func\u021biei criteriu J. A\u0219a cum am ar\u0103tat \u00een paragraful 1, aceast\u0103 valoarea nu trebuie s\u0103 fie nici prea mare, nici prea mic\u0103. P\u00e2n\u0103 acum nu s-a elaborat o metod\u0103 universal\u0103 de alegere a lui \u03c1 \u00eentr-o problem\u0103 dat\u0103. Se recomand\u0103 ca \u03c1 s\u0103 fie subunitar sau, eventual, descresc\u0103tor odat\u0103 cu cre\u0219terea num\u0103rului itera\u021biei.<br \/>\nS-a ar\u0103tat c\u0103 viteza de sc\u0103dere a func\u021biei criteriu depinde foarte puternic de valoarea aleas\u0103 pentru \u03c1. \u00cen general se recomand\u0103 s\u0103 se testeze evolu\u021bia re\u021belei pentru diverse valori ale lui \u03c1 \u0219i apoi s\u0103 se aleag\u0103 o valoarea convenabil\u0103. Se poate porni cu o valoare relativ mic\u0103 pentru \u03c1 \u0219i s\u0103 se creasc\u0103 aceast\u0103 valoare, at\u00e2t timp c\u00e2t prin aceast\u0103 cre\u0219tere scade J. \u00cen momentul \u00een care \u03c1 dep\u0103\u0219e\u0219te valoarea optim\u0103 \u0219i devine prea mare apar oscila\u021bii ale valorii lui J, care uneori cre\u0219te \u00een loc s\u0103 scad\u0103.<br \/>\nNoi am testat comportarea re\u021belei pentru diverse valori ale lui \u03c1 aleg\u00e2nd valoarea optim\u0103.<br \/>\nTotu\u0219i nu numai \u03c1 este neprecizat, a\u0219a cum p\u0103rea la prima vedere, ci \u0219i valorile ini\u021biale ale ponderilor re\u021belei. Acestea se aleg aleatoriu \u00een intervalul [-1, 1]. \u00cen concluzie, ar fi gre\u0219it s\u0103 se aleag\u0103 \u03c1 pentru un oarecare set de ponderi. De aceea am simulat comportarea fiec\u0103rei re\u021bele, pentru fiecare \u03c1, \u00een cazul a 10 seturi diferite de ponderi ini\u021biale. Este evident c\u0103, pentru o aceea\u0219i problem\u0103, indiferent de metoda de \u00eenv\u0103\u021bare \u0219i de parametrii variabili, am luat de fiecare dat\u0103 <strong>acelea\u0219i<\/strong> 10 seturi de ponderi aleatorii ini\u021biale.<br \/>\nFiecare metod\u0103 am rulat-o acela\u0219i timp. De asemenea, deoarece itera\u021biile nu dureaz\u0103 un timp egal chiar la aceea\u0219i metod\u0103, am reprezentat pe abscis\u0103 \u00een graficele din <strong>figura 3<\/strong> timpul de rulare (2 minute pe un PC-IBM 486; 50 MHz: dar acest timp este neesen\u021bial la compararea metodelor).<br \/>\nPe ordonat\u0103 am reprezentat logaritmul zecimal al func\u021biei criteriu J pentru \u00eentreaga mul\u021bime a exemplelor. Am ales reprezentarea la scar\u0103 logaritmic\u0103 pentru c\u0103 erorile ajung oricum la valori mult mai mici dec\u00e2t cele ini\u021biale, dar conteaz\u0103 ordinul de m\u0103rime al acestora la diverse momente de timp.<br \/>\nFiecare linie de 3 grafice prezint\u0103 pe scurt rezultatele simul\u0103rilor \u00een cazul unei probleme. Graficul din st\u00e2nga prezint\u0103\u00a0\u00a0<img title=\"lg \\underset{i=1,10}{\\underline{max}}J_{i}(t;\\rho )\" src=\"http:\/\/latex.codecogs.com\/gif.latex?lg&amp;space;\\underset{i=1,10}{\\underline{max}}J_{i}(t;\\rho&amp;space;)\" alt=\"\" \/>, unde i este setul de ponderi ini\u021biale pentru un \u03c1 dat la fiece moment de timp t. Graficul din dreapta este al func\u021biilor <img title=\"lg \\underset{i=1,10}{\\underline{min}}J_{i}(t;\\rho )\" src=\"http:\/\/latex.codecogs.com\/gif.latex?lg&amp;space;\\underset{i=1,10}{\\underline{min}}J_{i}(t;\\rho&amp;space;)\" alt=\"\" \/>, iar cel din mijloc este <img title=\"\\lg \\left [ \\frac{1}{10}\\cdot \\sum_{i=1}^{10}J_{i}(t;\\rho ) \\right ]\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\lg&amp;space;\\left&amp;space;[&amp;space;\\frac{1}{10}\\cdot&amp;space;\\sum_{i=1}^{10}J_{i}(t;\\rho&amp;space;)&amp;space;\\right&amp;space;]\" alt=\"\" \/>, deci corespunde mediei func\u021biilor criteriu pentru un anume \u03c1.<br \/>\nPentru a alege valoarea optim\u0103 a lui \u03c1 compar\u0103m curbele logaritmului mediilor (graficul din mijloc), iar \u00een cazul mai multor valori cu rezultate asem\u0103n\u0103toare, ne vom ghida \u0219i dup\u0103 celelalte grafice. Din motive de claritate a desenului nu am desenat curbele pentru toate valorile \u03c1 pentru care am efectuat simul\u0103rile.<br \/>\nS-au efectuat urm\u0103toarele simul\u0103ri:<br \/>\n1). <strong>BINAR<\/strong>: \u03c1=0.1 (curbele 1); \u03c1=0.18; \u03c1=0.26 (curbele 2); \u03c1=0.3; \u03c1=0.34 (curbele 3);<br \/>\n\u03c1=0.38; \u03c1=0.42 (curbele 4); \u03c1=0.5.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.34 (curbele 3)<\/strong>.<br \/>\n2). <strong>COUNTER<\/strong>: \u03c1=0.1; \u03c1=0.18 (curbele 1); \u03c1=0.26; \u03c1=0.34 (curbele 2); \u03c1=0.42; \u03c1=0.5<br \/>\n(curbele 3); \u03c1=0.58; \u03c1=0.66 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.5 (curbele 3)<\/strong>.<br \/>\n3). <strong>MULTIPLEXOR<\/strong>: \u03c1=0.1; \u03c1=0.18; \u03c1=0.26; \u03c1=0.34; \u03c1=0.42 (curbele 1); \u03c1=0.5; \u03c1=0.58;<br \/>\n\u03c1=0.66 (curbele 2); \u03c1=0.74; \u03c1=0.82; \u03c1=0.9 (curbele 3); \u03c1=0,98; \u03c1=1.06;<br \/>\n\u03c1=1.14; \u03c1=1.22 (curbele 4); \u03c1=1.3.<br \/>\nCel mai bun rezultat: <strong>\u03c1=1.22 (curbele 4)<\/strong>.<br \/>\n4). <strong>5\u00d75 TABLE<\/strong>: \u03c1=0.1 (curbele 1); \u03c1=0.18 (curbele 2); \u03c1=0.26; \u03c1=0.34 (curbele 3); \u03c1=0.42;<br \/>\n\u03c1=0.5; \u03c1=0.58 (curbele 4); \u03c1=0.66.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.58 (curbele 4)<\/strong>.<\/p>\n<p>5). <strong>ASSOCIATIVE MEMORY<\/strong>: \u03c1=0.1 (curbele 1); \u03c1=0.18 (curbele 2); \u03c1=0.26; \u03c1=0.34;<br \/>\n(curbele 3); \u03c1=0.42; \u03c1=0.5 (curbele 4); \u03c1=0.58.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.5 (curbele 4)<\/strong>.<br \/>\n6). <strong>FUNCTION<\/strong>: \u03c1=0.01 (curbele 1); \u03c1=0.02 (curbele 2); \u03c1=0.04; \u03c1=0.06 (curbele 3); \u03c1=0.1;<br \/>\n(curbele 4); \u03c1=0.18.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.02 (curbele 2)<\/strong>.<br \/>\n3.2. <span style=\"text-decoration: underline;\">BackPropagation with Term Proportion <strong>(BPTP)<\/strong><\/span><br \/>\nDeoarece, \u00een general, convergen\u021ba este lent\u0103 la regula backpropagation, s-au propus diverse \u00eembun\u0103t\u0103\u021biri ale algoritmului de modificare a ponderilor. La algoritmul backpropagation, procedura de \u00eenv\u0103\u021bare necesit\u0103 o modificare a ponderilor cu <img title=\"\\frac{ \\partial J_{s}^{\\mu} }{\\partial w}\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\frac{&amp;space;\\partial&amp;space;J_{s}^{\\mu}&amp;space;}{\\partial&amp;space;w}\" alt=\"\" \/>. Metoda gradientului negativ impunea luarea de pa\u0219i infinitezimali, constanta de propor\u021bionalitate fiind rata \u00eenv\u0103\u021b\u0103rii, \u03c1. Din motive practice, de convergen\u021b\u0103 mai rapid\u0103, alegem o rat\u0103 de \u00eenv\u0103\u021bare c\u00e2t mai mare f\u0103r\u0103 a ajunge la oscila\u021bii.<br \/>\nO cale de evitare a oscila\u021biilor la valori mari ale lui \u03c1 este de a modifica ponderea \u00een func\u021bie \u0219i de modificarea anterioar\u0103, prin ad\u0103ugarea unui <strong>termen propor\u021bie<\/strong>:<br \/>\n<img title=\"\\Delta w_{ji}(k+1)=\\rho \\cdot \\Delta _{j}^{\\mu}\\cdot x_{i}^{\\mu }+\\beta \\cdot \\Delta w_{ji}(k))(22)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;w_{ji}(k+1)=\\rho&amp;space;\\cdot&amp;space;\\Delta&amp;space;_{j}^{\\mu}\\cdot&amp;space;x_{i}^{\\mu&amp;space;}+\\beta&amp;space;\\cdot&amp;space;\\Delta&amp;space;w_{ji}(k))(22)\" alt=\"\" \/><br \/>\nunde k este num\u0103rul itera\u021biei \u0219i \u03b2 este o constant\u0103 pozitiv\u0103.<br \/>\n\u00cen literatur\u0103 se afirm\u0103 c\u0103 prin ad\u0103ugarea termenului propor\u021bie, minimul se ob\u021bine mai rapid pentru c\u0103 sunt permise rate mai mari de \u00eenv\u0103\u021bare f\u0103r\u0103 a avea oscila\u021bii. De asemenea se recomand\u0103 [7] ca \u03b2=\u03c1\/k.<br \/>\nPentru a testa aceste afirma\u021bii am antrenat re\u021belele pentru fiecare problem\u0103 pentru care am testat regula clasic\u0103 pornind de la acelea\u0219i 10 seturi de ponderi ini\u021biale ale fiec\u0103rei probleme.<br \/>\nLa fiecare problem\u0103 am crescut valoarea lui \u03c1 g\u0103sit\u0103 optim\u0103 cu algoritmul <strong>BP<\/strong> p\u00e2n\u0103 c\u00e2nd ap\u0103reau oscila\u021bii, determin\u00e2nd astfel \u03c1 optim \u0219i pentru acest algoritm.<br \/>\nAm rulat \u00een fiecare caz acela\u0219i timp ca \u0219i \u00een algoritmul <strong>BP<\/strong>, de\u0219i, fiind necesare mai multe calcule, num\u0103rul de itera\u021bii era mai mic, acesta fiind tributul pl\u0103tit pentru cre\u0219terea complexit\u0103\u021bii algoritmului.<br \/>\nRezultatul simul\u0103rilor sunt sintetizate \u00een <strong>figura 4,<\/strong> unde aranjarea graficelor este aceea\u0219i ca \u0219i \u00een <strong>figura 3<\/strong>:<br \/>\n1). <strong>BINAR<\/strong>: \u03c1=0.3 (curbele 3); \u03c1<sub>BP<\/sub>=0.34 (curbele 1); \u03c1<sub>BP<\/sub>=0.38 (curbele 2); \u03c1=0.42.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.38 (curbele 2)<\/strong>.<br \/>\n2). <strong>COUNTER<\/strong>: \u03c1=0.42 (curbele 1); \u03c1<sub>BP<\/sub>=0.5 (curbele 2); \u03c1=0.58 (curbele 3).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.5 (curbele 2)<\/strong>.<br \/>\n3). <strong>MULTIPLEXOR<\/strong>: \u03c1=0.9; \u03c1=0.98 (curbele 3); \u03c1=1.06; \u03c1=1.14; \u03c1<sub>BP<\/sub>=1.22 (curbele 2); \u03c1=1.3 (curbele 1).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.98 (curbele 3)<\/strong>.<br \/>\n4). <strong>5\u00d75 TABLE<\/strong>: \u03c1<sub>BP<\/sub>=0.58 (curbele 1); \u03c1=0.66 (curbele 2); \u03c1=0.74 (curbele 3); \u03c1=0.82.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.74 (curbele 3)<\/strong>.<br \/>\n5). <strong>ASSOCIATIVE MEMORY<\/strong>: \u03c1=0.34; \u03c1=0.42 (curbele 1); \u03c1<sub>BP<\/sub>=0.5 (curbele 2); \u03c1=0.58 (curbele 3).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.42 (curbele 1)<\/strong>.<br \/>\n6). <strong>FUNCTION<\/strong>: \u03c1=0.01 (curbele 1); \u03c1<sub>BP<\/sub>=0.02 (curbele 2); \u03c1=0.04 (curbele 3).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.02 (curbele 2)<\/strong>.<\/p>\n<p>3.3 <span style=\"text-decoration: underline;\">BackPropagation with Term Proportion and Restart (<strong>BPTPR<\/strong>)<\/span><br \/>\nPentru a asigura convergen\u021ba algoritmului <strong>BPTP<\/strong> am luat \u03b2=\u03c1\/k \u0219i, dup\u0103 un num\u0103r mare de itera\u021bii, \u03b2 devine at\u00e2t de mic \u00eenc\u00e2t termenul propor\u021bie nu mai are nicio contribu\u021bie la varia\u021bia ponderilor. De aceea s-a propus introducerea unui <strong>restart<\/strong> \u00een sensul c\u0103 se \u201drestarteaz\u0103\u201d periodic direc\u021bia de modificare a ponderilor la direc\u021bia gradientului. Astfel:<br \/>\n<img title=\"\\beta _{k}=\\left\\{\\begin{matrix} \\rho \/i \\: \\: pentru \\: i\\neq 0\\\\0\\: \\: \\: \\; \\: \\: pentru \\: i=0 \\end{matrix}\\right.\\; \\; unde i=k\\: mod\\: I\\; (23)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\beta&amp;space;_{k}=\\left\\{\\begin{matrix}&amp;space;\\rho&amp;space;\/i&amp;space;\\:&amp;space;\\:&amp;space;pentru&amp;space;\\:&amp;space;i\\neq&amp;space;0\\\\0\\:&amp;space;\\:&amp;space;\\:&amp;space;\\;&amp;space;\\:&amp;space;\\:&amp;space;pentru&amp;space;\\:&amp;space;i=0&amp;space;\\end{matrix}\\right.\\;&amp;space;\\;&amp;space;unde&amp;space;i=k\\:&amp;space;mod\\:&amp;space;I\\;&amp;space;(23)\" alt=\"\" \/><br \/>\nPentru I=1 metoda se reduce la BPTP. Se recomand\u0103 ca I s\u0103 ia valori \u00eentre 2 \u0219i 10.<br \/>\nDe data aceasta avem dou\u0103 variabile: \u03c1 \u0219i I. Presupun\u00e2nd c\u0103 valoarea optim\u0103 a lui \u03c1 nu se schimb\u0103 prea mult fa\u021b\u0103 de metoda anterioar\u0103, se caut\u0103 valoarea optim\u0103 a lui I \u021bin\u00e2nd \u03c1=\u03c1<sub>optim BPTP<\/sub> pentru fiecare problem\u0103 \u00een parte, dup\u0103 care fix\u00e2nd pe I, verific\u0103m optimul lui \u03c1 \u00een jurul valorii \u03c1<sub>optim BPTP<\/sub>.<br \/>\nEfectu\u00e2nd simul\u0103rile \u00een acelea\u0219i condi\u021bii ca \u0219i la metodele anterioare ob\u021binem rezultatele prezentate sintetic \u00een <strong>figura 5<\/strong>:<br \/>\n1). <strong>BINAR<\/strong>: \u03c1<sub>BPTP<\/sub>=0.38 \u0219i I=3; \u03c1<sub>BPTP<\/sub>=0.38 \u0219i I=5 (curbele 1);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.38 \u0219i I=6 (curbele 2);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.38 \u0219i I=7 (curbele 3);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.38 \u0219i I=10; I=6 \u0219i \u03c1=0.34; I=6 \u0219i \u03c1=0.42 (curbele 4); I=6 \u0219i \u03c1=0.46;<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.42 \u0219i I=6 (curbele 4)<\/strong>.<br \/>\n2). <strong>COUNTER<\/strong>: \u03c1<sub>BPTP<\/sub>=0.5 \u0219i I=3 (curbele 1);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.5 \u0219i I=4 (curbele 2);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.5 \u0219i I=5 (curbele 3);<br \/>\nI=4 \u0219i \u03c1=0.34; I=4 \u0219i \u03c1=0.42 (curbele 4); I=4 \u0219i \u03c1=0.58;<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.42 \u0219i I=4 (curbele 4)<\/strong>.<br \/>\n3). <strong>MULTIPLEXOR<\/strong>: \u03c1<sub>BPTP<\/sub>=0.98 \u0219i I=2 (curbele 1);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.98 \u0219i I=3 (curbele 2);<br \/>\nI=2 \u0219i \u03c1=0.9 (curbele 3);<br \/>\nI=2 \u0219i \u03c1=1.06 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.98 \u0219i I=2 (curbele 1)<\/strong>.<br \/>\n4). <strong>5\u00d75 TABLE<\/strong>: \u03c1<sub>BPTP<\/sub>=0.74 \u0219i I=2; \u03c1<sub>BPTP<\/sub>=0.74 \u0219i I=3 (curbele 1);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.74 \u0219i I=4; \u03c1<sub>BPTP<\/sub>=0.74 \u0219i I=5 (curbele 2);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.74 \u0219i I=6; \u03c1<sub>BPTP<\/sub>=0.74 \u0219i I=7 (curbele 3);<br \/>\nI=5 \u0219i \u03c1=0.66; I=5 \u0219i \u03c1=0.82 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.74 \u0219i I=5 (curbele 2)<\/strong>.<br \/>\n5). <strong>ASSOCIATIVE MEMORY<\/strong>: \u03c1<sub>BPTP<\/sub>=0.42 \u0219i I=2 (curbele 1);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.42 \u0219i I=3 (curbele 2);<br \/>\nI=2 \u0219i \u03c1=0.34 (curbele 3);<br \/>\nI=2 \u0219i \u03c1=0.5 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.42 \u0219i I=2 (curbele 1)<\/strong>.<br \/>\n6). <strong>FUNCTION<\/strong>: \u03c1<sub>BPTP<\/sub>=0.02 \u0219i I=2 (curbele 1);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.02 \u0219i I=3 (curbele 2);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.02 \u0219i I=4 (curbele 3);<br \/>\n\u03c1<sub>BPTP<\/sub>=0.02 \u0219i I=5; I=3 \u0219i \u03c1=0.01; I=3 \u0219i \u03c1=0.04 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.02 \u0219i I=3 (curbele 4)<\/strong>.<\/p>\n<p>3.4. <span style=\"text-decoration: underline;\">Conjugate Gradient BackPropagation (<strong>CGBP<\/strong>)<\/span><br \/>\nO alt\u0103 idee a fost aplicarea tehnicilor aprofundate \u00een teoria optimiz\u0103rii. De fapt, se observ\u0103 c\u0103 BPTPR este o variant\u0103 simplificat\u0103 a <strong>metodei gradien\u021bilor conjuga\u021bi cu restart<\/strong>. Metoda gradien\u021bilor conjuga\u021bi cu restart difer\u0103 de BPTPR numai prin alegerea lui \u03b2, care aici se calculeaz\u0103 mai complicat, \u00een func\u021bie de norma gradientului de la itera\u021bia actual\u0103, k, \u0219i de la itera\u021bia precedent\u0103, k-1, dup\u0103 formula:<br \/>\n<img title=\"\\beta _{k}=\\left\\{\\begin{matrix} \\frac{g_{i}}{g_{i-1}}\\cdot \\frac{g_{i}-g_{i-1}}{g_{i-1}}\\; pentru \\: i\\neq 0\\\\ 0 \\; \\;\\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; \\; pentru \\: i=0 \\end{matrix}\\right. unde\\; i - k\\: mod \\: I\\: \\; (24)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\beta&amp;space;_{k}=\\left\\{\\begin{matrix}&amp;space;\\frac{g_{i}}{g_{i-1}}\\cdot&amp;space;\\frac{g_{i}-g_{i-1}}{g_{i-1}}\\;&amp;space;pentru&amp;space;\\:&amp;space;i\\neq&amp;space;0\\\\&amp;space;0&amp;space;\\;&amp;space;\\;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;\\;&amp;space;pentru&amp;space;\\:&amp;space;i=0&amp;space;\\end{matrix}\\right.&amp;space;unde\\;&amp;space;i&amp;space;=&amp;space;k\\:&amp;space;mod&amp;space;\\:&amp;space;I\\:&amp;space;\\;&amp;space;(24)\" alt=\"\" \/><br \/>\nTotu\u0219i, \u0219i la aceast\u0103 metod\u0103 trebuie ales coeficientul \u03c1 \u0219i indicele de restart I. De aceea am reluat simul\u0103rile cu scopul de a determina valorile optime ale lui \u03c1 \u0219i I. Pentru \u00eenceput se men\u021bine constant\u0103 valoarea lui \u03c1 (\u03c1=\u03c1<sub>optim BPTPR<\/sub>) \u0219i se caut\u0103 valoarea optim\u0103 a lui I \u00een jurul valorii I<sub>optim BPTPR<\/sub>. Apoi fix\u0103m pe I la I<sub>optim CGBP<\/sub> \u0219i c\u0103ut\u0103m \u03c1<sub>optim CGBP<\/sub> \u00een jurul lui \u03c1<sub>optim BPTPR<\/sub>.<br \/>\nS-au efectuat urm\u0103toarele simul\u0103ri <strong>(figura 6)<\/strong>:<br \/>\n1). <strong>BINAR<\/strong>: \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=5 (curbele 1); \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I<sub>BPTPR<\/sub>= 6; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=7;<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=8 (curbele 2);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=9; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=10; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=11 (curbele 3);<br \/>\nI=11 \u0219i \u03c1=0.34; I=11 \u0219i \u03c1=0.5 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.42 \u0219i I=11 (curbele 3)<\/strong>.<br \/>\n2). <strong>COUNTER<\/strong>: \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=3 (curbele 1);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I<sub>BPTPR<\/sub>=4 (curbele 2);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=5 (curbele 3);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=6; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=7; I=4 \u0219i \u03c1=0.5 (curbele 4);<br \/>\nI=4 \u0219i \u03c1=0.58.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.5 \u0219i I=4 (curbele 4)<\/strong>.<br \/>\n3). <strong>MULTIPLEXOR<\/strong>: \u03c1<sub>BPTPR<\/sub>=0.98 \u0219i I=4 (curbele 1);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.98 \u0219i I=5 (curbele 2);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.98 \u0219i I=6 (curbele 3);<br \/>\nI=5 \u0219i \u03c1=1.06; I=5 \u0219i \u03c1=1.14; I=5 \u0219i \u03c1=1.22; I=5 \u0219i \u03c1=1.3 (curbele 4);<br \/>\nI=5 \u0219i \u03c1=1.38.<br \/>\nCel mai bun rezultat: <strong>\u03c1=1.3 \u0219i I=5 (curbele 4)<\/strong>.<br \/>\n4). <strong>5\u00d75 TABLE<\/strong>: \u03c1<sub>BPTPR<\/sub>=0.74 \u0219i I=2 (curbele 1);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.74 \u0219i I=3 (curbele 2);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.74 \u0219i I=4 (curbele 3);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.74 \u0219i I<sub>BPTPR<\/sub>=5; \u03c1<sub>BPTPR<\/sub>=0.74 \u0219i I=6; I=3 \u0219i \u03c1=0.66; I=3 \u0219i \u03c1=0.82 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.74 \u0219i I=3 (curbele 2)<\/strong>.<\/p>\n<p>5). <strong>ASSOCIATIVE MEMORY<\/strong>: \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=4; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=5 (curbele 1);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=6; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=7 (curbele 2);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=8; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=9; \u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=10(curbele 3);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.42 \u0219i I=11; I=10 \u0219i \u03c1=0.34; I=11 \u0219i \u03c1=0.5 (curbele 4).<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.42 \u0219i I=10 (curbele 3)<\/strong>.<br \/>\n6). <strong>FUNCTION<\/strong>: \u03c1<sub>BPTPR<\/sub>=0.02 \u0219i I=2 (curbele 1);<br \/>\n\u03c1<sub>BPTPR<\/sub>=0.02 \u0219i I<sub>BPTPR<\/sub>=3; I=2 \u0219i \u03c1=0.06; I=2 \u0219i \u03c1=0.1 (curbele 2);<br \/>\nI=2 \u0219i \u03c1=0.18; \u03c1=0.1 \u0219i I=3 (curbele 3);<br \/>\n\u03c1=0.1 \u0219i I=4; \u03c1=0.1 \u0219i I=5; \u03c1=0.1 \u0219i I=6 (curbele 4); \u03c1=0.1 \u0219i I=7.<br \/>\nCel mai bun rezultat: <strong>\u03c1=0.1 \u0219i I=6 (curbele 4)<\/strong>.<\/p>\n<p><span style=\"text-decoration: underline;\">Observa\u021bie<\/span>: Compar\u00e2nd rezultatele ob\u021binute \u00een variantele de \u00eenv\u0103\u021bare optime prin aceste 4 metode se observ\u0103 \u00een <strong>figura 7<\/strong> (curba 1: BP; curba 2: BPTP; curba 3: BPTPR; curba 4: CGBP) superioritatea deta\u0219at\u0103 a metodei CGBP. Situa\u021bia ap\u0103rut\u0103 uneori (de exemplu, cazul problemei COUNTER) unde, de\u0219i pentru unele ponderi ini\u021biale ob\u021binem rezultate foarte bune, uneori rezultatele sunt mai slabe ca la celelalte metode, apare din dou\u0103 motive:<br \/>\na)complexitatea calculelor fiind mai mare, la acela\u0219i timp de rulare al algoritmului se execut\u0103 mai pu\u021bine itera\u021bii de \u00eenv\u0103\u021bare;<br \/>\nb)probabil c\u0103 timpul de rulare a fost prea scurt pentru ca tendin\u021ba descresc\u0103toare a algoritmului CGBP s\u0103 reu\u0219easc\u0103 s\u0103 coboare func\u021bia criteriu la valori mai mici.<\/p>\n<p>3.5. <span style=\"text-decoration: underline;\"><strong>BP<\/strong> and <strong>CGBP<\/strong> using the error\u2019s absolute value minimization criterion<\/span><\/p>\n<p>\u00cen toate exemplele prezentate aici, ponderile au fost calculate \u00een termeni de eroare medie p\u0103tratic\u0103. Se prezint\u0103 urm\u0103toarea problem\u0103: care ar fi comportamentul re\u021belei dac\u0103 \u00een loc de eroarea p\u0103tratic\u0103 s-ar considera valoarea de eroare absolut\u0103? Am studiat aceast\u0103 problem\u0103 \u00een cazul a dou\u0103 metode de \u00eenv\u0103\u021bare: BP \u0219i CGBP, care s-au dovedit a fi cele mai bune.<\/p>\n<p>Eroarea total\u0103 va fi calculat\u0103 folosind aceea\u0219i formul\u0103 <strong>(3)<\/strong>, dar <img title=\"J_{s}^{\\mu }\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;J_{s}^{\\mu&amp;space;}\" alt=\"\" \/>, eroarea de ie\u0219ire absolut\u0103 pentru modelul \u03bc, va fi:<\/p>\n<p><img title=\"J_{s}^{\\mu }=\\frac{1}{2}\\cdot \\sum_{m=1}^{M}\\left | d_{m}^{\\mu }- \\mathfrak{F}\\left ( a_{m}^{\\mu } \\right ) \\right | (25)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?J_{s}^{\\mu&amp;space;}=\\frac{1}{2}\\cdot&amp;space;\\sum_{m=1}^{M}\\left&amp;space;|&amp;space;d_{m}^{\\mu&amp;space;}-&amp;space;\\mathfrak{F}\\left&amp;space;(&amp;space;a_{m}^{\\mu&amp;space;}&amp;space;\\right&amp;space;)&amp;space;\\right&amp;space;|&amp;space;(25)\" alt=\"\" \/><\/p>\n<p>Unde, ca \u0219i \u00een cazul precedent, d ^ \u03bc este vectorul de ie\u0219ire dorit pentru clasa \u03bc.<\/p>\n<p>Urm\u00e2nd exact acela\u0219i ra\u021bionament prezentat pe larg de formulele <strong>(5) &#8211; (10),<\/strong> vor rezulta rela\u021biile de aplicat \u00een acest caz:<\/p>\n<p><img title=\"\\Delta q_{mj}=\\frac{\\rho }{2}\\cdot \\sum_{\\mu }\\sigma \\left [ d_{m}^{\\mu}-\\mathfrak{F(a_{m}^{\\mu}}) \\right ]\\cdot\\mathfrak{F(a_{m}^{\\mu})}\\cdot\\frac{\\partial a_{m}^{\\mu}}{\\partial q_{mj}}=\\frac{\\rho }{2}\\cdot \\sum_{\\mu }\\Delta _{m}^{\\mu }\\cdot v_{j}^{\\mu } (26)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;q_{mj}=\\frac{\\rho&amp;space;}{2}\\cdot&amp;space;\\sum_{\\mu&amp;space;}\\sigma&amp;space;\\left&amp;space;[&amp;space;d_{m}^{\\mu}-\\mathfrak{F(a_{m}^{\\mu}})&amp;space;\\right&amp;space;]\\cdot\\mathfrak{F(a_{m}^{\\mu})}\\cdot\\frac{\\partial&amp;space;a_{m}^{\\mu}}{\\partial&amp;space;q_{mj}}=\\frac{\\rho&amp;space;}{2}\\cdot&amp;space;\\sum_{\\mu&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;}\\cdot&amp;space;\\nu_{j}^{\\mu&amp;space;}&amp;space;(26)\" alt=\"\" \/><\/p>\n<p><img title=\"\\Delta s_{m}= -\\frac{\\rho }{2}\\sum_{\\mu }\\Delta _{m}^{\\mu } (27)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;S_{m}=&amp;space;-\\frac{\\rho&amp;space;}{2}\\sum_{\\mu&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;}&amp;space;(27)\" alt=\"\" \/><\/p>\n<p><img title=\"\\Delta _{m}^{\\mu }=\\sigma\\left[d_{m}^{\\mu}-\\mathfrak{F(a_{m}^{\\mu}})\\right]\\cdot\\mathfrak{F(a_{m}^{\\mu})}(28)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;_{m}^{\\mu&amp;space;}=\\sigma\\left[d_{m}^{\\mu}-\\mathfrak{F(a_{m}^{\\mu}})\\right]\\cdot\\mathfrak{F(a_{m}^{\\mu})}(28)\" alt=\"\" \/><\/p>\n<p>Unde, prin \u03c3[expresie] am considerat func\u021bia care \u00eenapoiaz\u0103 semnul expresiei date.<\/p>\n<p>Proced\u0103m similar pentru parametrii aloca\u021bi conexiunilor sinaptice dintre intrare \u0219i stratul ascuns:<strong> <\/strong><\/p>\n<p><img title=\"\\Delta w_{ji}=\\frac{\\rho }{2}\\cdot \\sum_{\\mu }\\overline{\\Delta}_{j}^{\\mu }\\cdot x_{i}^{\\mu }(29)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;w_{ji}=\\frac{\\rho&amp;space;}{2}\\cdot&amp;space;\\sum_{\\mu&amp;space;}\\overline{\\Delta}_{j}^{\\mu&amp;space;}\\cdot&amp;space;x_{i}^{\\mu&amp;space;}(29)\" alt=\"\" \/><\/p>\n<p><img title=\"\\Delta c_{j}=\\frac{\\rho }{2}\\cdot \\sum_{\\mu }\\overline{\\Delta }_{j}^{\\mu } (30)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;c_{j}=\\frac{\\rho&amp;space;}{2}\\cdot&amp;space;\\sum_{\\mu&amp;space;}\\overline{\\Delta&amp;space;}_{j}^{\\mu&amp;space;}&amp;space;(30)\" alt=\"\" \/><\/p>\n<p><img title=\"\\overline{\\Delta _{J}^{\\mu }}=\\left [ \\sum_{m}^{ }\\Delta _{m}^{\\mu }\\cdot q_{mj} \\right ]\\cdot \\mathfrak{F(\\overline{a_{j}^{\\mu }})}(10)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\dpi{120}&amp;space;\\overline{\\Delta&amp;space;_{J}^{\\mu&amp;space;}}=\\left&amp;space;[&amp;space;\\sum_{m}^{&amp;space;}\\Delta&amp;space;_{m}^{\\mu&amp;space;}\\cdot&amp;space;q_{mj}&amp;space;\\right&amp;space;]\\cdot&amp;space;\\mathfrak{F(\\overline{a_{j}^{\\mu&amp;space;}})}(31)\" alt=\"\" \/><\/p>\n<p>Extinz\u00e2nd la o re\u021bea feed-forward cu R-straturi, criteriul de eroare care ar trebui minimizat este valoarea erorii absolute, determinat\u0103 pe setul tuturor exemplelor de antrenare:<\/p>\n<p><img title=\"J_{p}=\\frac{1}{2}\\cdot \\sum_{\\mu }\\sum_{j_{1}=1}^{H_{1}}\\left | d_{j_{1}}^{\\mu }-v_{j_{1}}^{\\mu } \\right | (32)\" src=\"http:\/\/latex.codecogs.com\/gif.latex?J_{p}=\\frac{1}{2}\\cdot&amp;space;\\sum_{\\mu&amp;space;}\\sum_{j_{1}=1}^{H_{1}}\\left&amp;space;|&amp;space;d_{j_{1}}^{\\mu&amp;space;}-\\nu_{j_{1}}^{\\mu&amp;space;}&amp;space;\\right&amp;space;|&amp;space;(32)\" alt=\"\" \/><\/p>\n<p>Expresiile <strong>(13-16)<\/strong> pot fi u\u0219or modificate pentru cazul de fa\u021b\u0103; modificarea major\u0103 e legat\u0103 de formula <strong>(15)<\/strong>, care devine:<\/p>\n<p><img title=\"\\Delta _{j_{1}}^{\\mu }=\\sum_{j_{1}=1}^{H_{1}}\\sigma \\left [ d_{j_{1}}^{\\mu }-\\nu _{j_{1}}^{\\mu } \\right ]\\cdot \\mathfrak{F}\\left ( \\nu _{j_{1}}^{\\mu } \\right )(33))\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\Delta&amp;space;_{j_{1}}^{\\mu&amp;space;}=\\sum_{j_{1}=1}^{H_{1}}\\sigma&amp;space;\\left&amp;space;[&amp;space;d_{j_{1}}^{\\mu&amp;space;}-\\nu&amp;space;_{j_{1}}^{\\mu&amp;space;}&amp;space;\\right&amp;space;]\\cdot&amp;space;\\mathfrak{F}\\left&amp;space;(&amp;space;\\nu&amp;space;_{j_{1}}^{\\mu&amp;space;}&amp;space;\\right&amp;space;)(33)\" alt=\"\" \/><\/p>\n<p>Pentru acelea\u0219i probleme pentru care au fost testate metodele de \u00eenv\u0103\u021bare anterioare, am rulat programe de simulare, iar rezultatele lor au fost reprezentate grafic, \u00eempreun\u0103 cu cel mai bun rezultat ob\u021binut \u00een cazul minimiz\u0103rii erorii p\u0103tratice. Spre deosebire de simul\u0103rile anterioare, aici am folosit doar un set de ponderi ini\u021biale \u0219i am p\u0103strat pentru reprezentarea grafic\u0103 doar cel mai bun rezultat. Acest fapt nu influen\u021beaz\u0103 concluziile finale privind compararea eficien\u021bei celor dou\u0103 metode, se poate vedea, \u00een cele mai multe cazuri, superioritatea metodei folosind criteriul de minimizare a erorii p\u0103tratice fa\u021b\u0103 de a doua metod\u0103.<\/p>\n<p>Pe fiecare din cele 6 grafice (<strong>fig.6a-6f<\/strong>),<strong> <\/strong>curbele 1 \u0219i 2 reprezint\u0103 cel mai bun rezultat ob\u021binut cu <strong>BP<\/strong>, respectiv metoda <strong>CGBP<\/strong>, folosind cel de-al doilea criteriu de minimizare a erorii; iar curbele 3 \u0219i 4 reprezint\u0103 cel mai bun rezultat dup\u0103 aplicarea acelora\u0219i metode de \u00eenv\u0103\u021bare (<strong>BP<\/strong>, respectiv <strong>CGBP<\/strong>), dar pentru primul criteriu de minimizare a erorii (minimizarea erorii p\u0103tratice).<\/p>\n<p><a rel=\"attachment wp-att-488\" href=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?attachment_id=488\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-488\" title=\"fig6a\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6a1-300x186.png\" alt=\"\" width=\"300\" height=\"186\" srcset=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6a1-300x186.png 300w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6a1-1024x636.png 1024w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><a rel=\"attachment wp-att-489\" href=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?attachment_id=489\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-489\" title=\"fig6b\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6b1-300x179.png\" alt=\"\" width=\"300\" height=\"179\" srcset=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6b1-300x179.png 300w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6b1-1024x612.png 1024w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p><a rel=\"attachment wp-att-490\" href=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?attachment_id=490\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-490\" title=\"fig6c\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6c1-300x191.png\" alt=\"\" width=\"300\" height=\"191\" srcset=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6c1-300x191.png 300w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6c1-1024x652.png 1024w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><a rel=\"attachment wp-att-491\" href=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?attachment_id=491\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-491\" title=\"fig6d\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6d1-300x179.png\" alt=\"\" width=\"300\" height=\"179\" srcset=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6d1-300x179.png 300w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6d1-1024x613.png 1024w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6d1.png 2028w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p><a rel=\"attachment wp-att-492\" href=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?attachment_id=492\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-492\" title=\"fig6e\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6e1-300x171.png\" alt=\"\" width=\"300\" height=\"171\" srcset=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6e1-300x171.png 300w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6e1-1024x585.png 1024w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6e1.png 2028w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><a rel=\"attachment wp-att-493\" href=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/?attachment_id=493\"><img loading=\"lazy\" class=\"alignnone size-medium wp-image-493\" title=\"fig6f\" src=\"http:\/\/webspace.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6f1-300x180.png\" alt=\"\" width=\"300\" height=\"180\" srcset=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6f1-300x180.png 300w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6f1-1024x617.png 1024w, https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/wp-content\/uploads\/fig6f1.png 1996w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Pentru metoda <strong>BP <\/strong>(curbele 1), am pornit ini\u021bial de la cea mai favorabil\u0103 valoare pentru \u03c1, pentru care am ob\u021binut cel mai bun comportament al re\u021belei, a\u0219a cum se arat\u0103 la punctul 3. Am variat \u03c1 p\u00e2n\u0103 c\u00e2nd s-a atins cel mai bun comportament pentru criteriul erorii absolute, s-a reprezentat grafic \u00een curbele 1, acestea trebuie comparate cu curbele 3 (<strong>BP<\/strong> pentru criteriul erorii p\u0103tratice).<\/p>\n<p>Pentru metoda <strong>CGBP<\/strong> (curbele 2), parametrii ini\u021biali au fost: \u03c1 optim \u00a0g\u0103sit anterior \u0219i indicele optim de repornire I<sub>CGBP optim<\/sub>, dar folosind metoda de eroare p\u0103tratic\u0103. Coeficientul \u03c1 a fost men\u021binut constant \u0219i indicele I a fost variat; apoi, dup\u0103 alegerea unui I optim, acesta a fost men\u021binut constant \u0219i \u03c1 a fost variat p\u00e2n\u0103 c\u00e2nd s-a atins cea mai potrivit\u0103 valoare. Cu ace\u0219ti doi parametri cei mai buni, pentru fiecare aplica\u021bie au fost reprezentate graficele, curbele 2 trebuie comparate cu curbele 4<span style=\"text-decoration: underline;\">.<\/span><\/p>\n<p>Urm\u0103toarele simul\u0103ri au fost efectuate:<\/p>\n<p>1).\u00a0<strong>BINAR: BP:<\/strong> \u03c1=0.66 (curbele 1); <strong>CGBP:<\/strong> \u03c1=0.82, I=11 (curbele 2);<strong>BP: \u03c1<\/strong>=0.34 (curbele 3); <strong>CGBP: \u03c1<\/strong>=0.42, I=11 (curbele 4).<\/p>\n<p>2).\u00a0<strong>COUNTER: BP: <\/strong>\u03c1=0.5 (curbele 1); <strong>CGBP:<\/strong> \u03c1=0.5, I=4 (curbele 2);<strong>BP: \u03c1<\/strong>=0.5 (curbele 3); <strong>CGBP: \u03c1<\/strong>=0.42, I=4 (curbele 4).<\/p>\n<p>3).\u00a0<strong>MULTIPLEXOR: BP:<\/strong> \u03c1=1.22 (curbele 1); <strong>CGBP:<\/strong> \u03c1=1.22, I=5 (curbele 2);<strong>BP: \u03c1<\/strong>=1.22 (curbele 3); <strong>CGBP: \u03c1<\/strong>=1.3, I=5 (curbele 4).<\/p>\n<p>4).\u00a0<strong>5X5 TABLE: BP:<\/strong> \u03c1=0.58 (curbele 1); <strong>CGBP:<\/strong> \u03c1=0.58, I=4 (curbele 2);<strong>BP: \u03c1<\/strong>=0.58 (curbele 3); <strong>CGBP: \u03c1<\/strong>=0.74, I=3 (curbele 4).<\/p>\n<p>5).\u00a0<strong>ASSOCIATIVE MEMORY:<\/strong> \u03c1=0.5 (curbele 1); <strong>CGBP:<\/strong> \u03c1=0.5, I=10 (curbele 2);<strong>BP: \u03c1<\/strong>=0.5 (curbele 3); <strong>CGBP: \u03c1<\/strong>=0.42, I=10 (curbele 4).<\/p>\n<p>6).<strong>FUNCTION: BP:<\/strong> \u03c1=0.0005 (curbele 1); <strong>CGBP:<\/strong> \u03c1=0.5, I=10 (curbele 2);<strong>BP: \u03c1<\/strong>=0.02 (curbele 3); <strong>CGBP: \u03c1<\/strong>=0.1, I=6 (curbele 4).<\/p>\n<p>4. <span style=\"text-decoration: underline;\">O nou\u0103 metod\u0103 de tip <strong>BP<\/strong> (<strong>NBP<\/strong>)<\/span><\/p>\n<p>Metodele cele mai populare de modificare a ponderilor re\u021belelor feedforward, respectiv metodele de tip backpropagation, sunt de fapt metode de gradient. Deoarece gradientul este o m\u0103sur\u0103 local\u0103, rezult\u0103 c\u0103 pasul pe direc\u021bia gradientului, care pas este dat de rata de \u00eenv\u0103\u021bare \u03c1, trebuie s\u0103 fie infinitezimal, ceea ce implic\u0103 alegerea unui \u03c1 foarte mic. Dar aceasta ar conduce la o convergen\u021b\u0103 foarte \u00eenceat\u0103. De aceea, se alege totu\u0219i un \u03c1 mai mare. Pe de alt\u0103 parte, un \u03c1 prea mare poate conduce la oscila\u021bii foarte mari ale func\u021biei obiectiv. \u00cen plus, valorile lui \u03c1 sunt relative la problema de rezolvat.<br \/>\nO solu\u021bie ar putea fi alegerea unui \u03c1 oarecare \u0219i testarea lui pentru a vedea dac\u0103 produce oscila\u021bii \u0219i dac\u0103 da, atunci s\u0103 se mai scad\u0103 valoarea lui p\u00e2n\u0103 c\u00e2nd nu mai avem oscila\u021bii. Cu alte cuvinte, am putea alege un \u03c1 c\u00e2t mai mare posibil f\u0103r\u0103 apari\u021bia de oscila\u021bii. Dar acest tip de abordare conduce la un num\u0103r neprecizat de \u00eencerc\u0103ri pe un num\u0103r oarecare de itera\u021bii pentru a seta o valoare optim\u0103 a lui \u03c1. \u021ain\u00e2nd cont \u0219i de faptul c\u0103, dup\u0103 un num\u0103r oarecare de itera\u021bii se poate ajunge \u00eentr-o regiune \u00een spa\u021biul ponderilor unde respectiva valoare s\u0103 nu fie optim\u0103, Silva \u0219i Almeida [16] au ar\u0103tat avantajele alegerii unui pas independent pentru fiecare pondere \u00een re\u021bea. \u00cen algoritmul lor, rata de \u00eenv\u0103\u021bare se adapteaz\u0103 dup\u0103 fiecare pattern de antrenament:<br \/>\n<img title=\"\\rho _{ij}(k+1)=\\left\\{\\begin{matrix} u\\cdot \\rho _{ij}(k), &amp; daca\\: \\frac{\\partial J_{k+1}}{\\partial w_{ij}} si \\frac{\\partial J_{k}}{\\partial w_{ij}} \\: au \\: acelasi \\: semn\\\\ d\\cdot \\rho _{ij}(k),&amp; daca\\: \\frac{\\partial J_{k+1}}{\\partial w_{ij}} si \\frac{\\partial J_{k}}{\\partial w_{ij}} \\: au \\: semne\\: diferite \\end{matrix}\\right.\\; (25))\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\rho&amp;space;_{ij}(k+1)=\\left\\{\\begin{matrix}&amp;space;u\\cdot&amp;space;\\rho&amp;space;_{ij}(k),&amp;space;&amp;&amp;space;daca\\:&amp;space;\\frac{\\partial&amp;space;J_{k+1}}{\\partial&amp;space;w_{ij}}&amp;space;si&amp;space;\\frac{\\partial&amp;space;J_{k}}{\\partial&amp;space;w_{ij}}&amp;space;\\:&amp;space;au&amp;space;\\:&amp;space;acelasi&amp;space;\\:&amp;space;semn\\\\&amp;space;d\\cdot&amp;space;\\rho&amp;space;_{ij}(k),&amp;&amp;space;daca\\:&amp;space;\\frac{\\partial&amp;space;J_{k+1}}{\\partial&amp;space;w_{ij}}&amp;space;si&amp;space;\\frac{\\partial&amp;space;J_{k}}{\\partial&amp;space;w_{ij}}&amp;space;\\:&amp;space;au&amp;space;\\:&amp;space;semne\\:&amp;space;diferite&amp;space;\\end{matrix}\\right.\\;&amp;space;(34)\" alt=\"\" \/><br \/>\nunde u \u0219i d sunt constante pozitive: u pu\u021bin mai mare dec\u00e2t 1 (de exemplu 1,1) \u0219i d pu\u021bin mai mic\u0103 dec\u00e2t 1 (de exemplu 0,9). Ideea este de a descre\u0219te rata de \u00eenv\u0103\u021bare la apari\u021bia oscila\u021biilor. S-au mai propus \u0219i alte metode la care pasul variaz\u0103 la fiecare itera\u021bie [6,9].<br \/>\nO solu\u021bie mai bun\u0103 ar fi determinarea lui \u03c1 la fiecare itera\u021bie. Deci, vom alege la fiecare itera\u021bie un pas c\u00e2t mai mare posibil f\u0103r\u0103 a avea oscila\u021bii. Vom genera o valoarea aleatorie a lui \u03c1, \u0219i vom determina, cu metoda gradien\u021bilor conjuga\u021bi cu restart, valorile ponderilor \u0219i vom testa m\u0103sura erorii J. Dac\u0103 eroarea este mai mic\u0103 dec\u00e2t eroarea anterioar\u0103 vom mic\u0219ora \u03c1 p\u00e2n\u0103 c\u00e2nd eroarea devine mai mic\u0103 dec\u00e2t aceasta din urm\u0103. \u00cen acest fel, dac\u0103 pe direc\u021bia gradientului eroarea total\u0103 scade, alegem pasul \u00een acea direc\u021bie c\u00e2t mai mare posibil; \u00een caz contrar urm\u00e2nd s\u0103 folosim propriet\u0103\u021bile locale prin alegerea unui pas mic.<br \/>\nPrin posibilitatea de a alege pa\u0219i oric\u00e2t de mari se creeaz\u0103 premisele evad\u0103rii din minimele locale. A\u0219adar, vom evada dintr-un minim local dac\u0103 pe direc\u021bia gradientului se g\u0103se\u0219te un minim \u201dmai minim\u201d \u0219i \u00eent\u00e2mpl\u0103tor (datorit\u0103 alegerii aleatorie a \u03c1-ului ini\u021bial) d\u0103m peste el.<br \/>\nDeci, este foarte important\u0103 alegerea \u03c1-ului ini\u021bial din fiecare itera\u021bie, care trebuie s\u0103 fie suficient de mare pentru a putea evada din minimele locale, dar dac\u0103 este prea mare, atunci la fiecare itera\u021bie ar repeta pa\u0219ii de mic\u0219orare a pasului. De aceea propun urm\u0103toarea formul\u0103 [17] pe care am folosit-o \u00een simul\u0103ri:<\/p>\n<p>\u03c1(k+1)=(1\/2+\u03be)\u2219\u03c1<sub>mean<\/sub> ()\u2219(n<sub>iterat<\/sub>+1) (35)<br \/>\nunde \u03be este un num\u0103r uniform aleatoriu \u00een intervalul (0,1), n<sub>iterat<\/sub> este un \u00eentreg care indic\u0103 de c\u00e2te itera\u021bii criteriul J scade cu mai pu\u021bin de 1\u2030 \u0219i \u03c1<sub>mean<\/sub> este media aritmetic\u0103 a valorilor lui \u03c1 din ultimele 10 itera\u021bii. Justificarea acestei formule este:<br \/>\n&#8211; \u03c1 ini\u021bial trebuie s\u0103 fie cel pu\u021bin 0,5;<br \/>\n&#8211; pentru a minimiza timpul de c\u0103utare al lui \u03c1 final \u00een fiecare itera\u021bie este de presupus c\u0103 acesta va avea o valoare asem\u0103n\u0103toare cu valorile anterioare, pentru c\u0103 \u201drelieful\u201d func\u021biei criteriu J nu se schimb\u0103 substan\u021bial dac\u0103 pa\u0219ii exploateaz\u0103 propriet\u0103\u021bile locale ale mediului. De aceea \u03c1 ini\u021bial se determin\u0103 ca media valorilor finale ale lui \u03c1 \u00een ultimele 10 itera\u021bii. Pentru calcului mediei elimin\u0103m acele valorii care sunt prea mari sau prea mici \u00een raport cu celelalte. La \u00eenceputul \u00eenv\u0103\u021b\u0103rii \u03c1<sub>mean<\/sub> se seteaz\u0103 la valoarea de 0.1. Astfel:<br \/>\n<img title=\"\\rho _{m(k)}=exp\\left [ \\frac{\\sum_{i=1}^{10}ln(1+\\rho (k-i+1))}{10} \\right ]-1\" src=\"http:\/\/latex.codecogs.com\/gif.latex?\\rho&amp;space;_{m(k)}=exp\\left&amp;space;[&amp;space;\\frac{\\sum_{i=1}^{10}ln(1+\\rho&amp;space;(k-i+1))}{10}&amp;space;\\right&amp;space;]-1\" alt=\"\" \/><br \/>\n<strong>no<sub>\u03c1<\/sub> \u2236=10; \u03c1<sub>sum<\/sub> \u2236= 0;<\/strong><br \/>\n<strong> for i:=1 to 10 do<\/strong><br \/>\n<strong> if ((\u03c1(k-i+1) &gt; \u03c1<sub>m<\/sub>(k)\/10) and (\u03c1(k-i+1) &lt;10* \u03c1<sub>m<\/sub>(k))<\/strong><br \/>\n<strong> then \u03c1<sub>sum<\/sub>:=\u03c1<sub>sum<\/sub>+\u03c1(k-i+1)<\/strong><br \/>\n<strong> else no<sub>\u03c1<\/sub>:=no<sub>\u03c1<\/sub>-1;<\/strong><br \/>\n<strong> \u03c1<sub>mean<\/sub>(k):=\u03c1<sub>sum\/<\/sub>no<sub>\u03c1<\/sub>.<\/strong><br \/>\n&#8211; dac\u0103 eroarea J nu scade de la o itera\u021bie la alta, \u00eenseamn\u0103 c\u0103 suntem \u00eentr-un minim local, din care ar fi bine s\u0103 evad\u0103m, a\u0219a c\u0103 ar fi indicat s\u0103 m\u0103rim \u03c1 \u0219i de aceea apare ultimul factor. M\u0103rimea ini\u021bial\u0103 a lui \u03c1 fiind stabil\u0103, r\u0103m\u00e2ne de fixat modalitatea de m\u0103rire \u0219i mic\u0219orare a ratei de \u00eenv\u0103\u021bare:<br \/>\n1). Determin\u0103m cu algoritmul anterior un \u03c1 ini\u021bial;<br \/>\ni:=0;<br \/>\n2). i:=i+1<br \/>\n3). Calcul\u0103m:<br \/>\n-varia\u021bia ponderilor \u2206w cu formula gradien\u021bilor conjuga\u021bi cu restart: (22) \u0219i<br \/>\n(24) unde I=5<br \/>\n-ponderile: w<sub>k+1<\/sub>:=w<sub>k<\/sub>+\u03c1\u2219\u2206w<br \/>\n-cu noile ponderi determin\u0103m J<sub>1<\/sub><br \/>\n4). If J<sub>1<\/sub>&lt;J<sub>0<\/sub><br \/>\n<em>Then Repeat<\/em><br \/>\na). i:=i+1<br \/>\nb). Calcul\u0103m:<br \/>\n-varia\u021bia ponderilor \u2206w cu formula gradien\u021bilor conjuga\u021bi cu restart: (22) \u0219i (24)<br \/>\n-ponderile: w<sub>k+1<\/sub>:=w<sub>k<\/sub>+\u03c1\u2219u<sub>i<\/sub>\u2219\u2206w<br \/>\n-cu noile ponderi determin\u0103m J<sub>i<\/sub><br \/>\n<em>Until<\/em> J<sub>i<\/sub>&gt;J<sub>i+1<\/sub><br \/>\n<strong>w<sub>k+1<\/sub>:=w<sub>k<\/sub>+\u03c1\u2219u<sub>i-1<\/sub>\u2219\u2206w<\/strong><br \/>\n<em>Else Repeat<\/em><br \/>\na). i:=i+1<br \/>\nb). Calcul\u0103m:<br \/>\n-varia\u021bia ponderilor \u2206w cu formula gradien\u021bilor conjuga\u021bi cu restart<br \/>\n-ponderilor: w<sub>k+1<\/sub>=w<sub>k<\/sub>+(\u03c1\/u<sub>i<\/sub> )\u2219\u2206w<br \/>\n-cu noile ponderi determin\u0103m Ji<br \/>\n<em>Until<\/em> J<sub>i<\/sub>&lt;J<sub>0<\/sub><br \/>\n<strong>w<sub>k+1<\/sub>:=w<sub>k<\/sub>+(\u03c1\/u<sub>i<\/sub> ) \u2219\u2206w<\/strong><br \/>\n5). Trecem la itera\u021bia urm\u0103toare:<br \/>\n\u0218irul u<sub>i<\/sub> poate fi \u0219irul puterilor lui 2 (u<sub>i<\/sub>=2<sup>i<\/sup>), dar noi prefer\u0103m \u0219irul lui Fibonacci (1, 1, 2, 3, 5, 8, &#8230;), adic\u0103 u<sub>i<\/sub> = u<sub>i-1<\/sub>+u<sub>i+2<\/sub>, datorit\u0103 bunelor rezultate ob\u021binute cu acesta \u0219i \u00een teoria optimiz\u0103rii.<br \/>\nSe observ\u0103 din grafice c\u0103, indiferent de tipul problemei, eroarea scade cu c\u00e2teva ordine de m\u0103rime mai mult dec\u00e2t \u00een celelalte metode \u201dclasice\u201d. Chiar mai mult, eroarea evit\u0103 (sau evadeaz\u0103) minimele locale, tinz\u00e2nd c\u0103tre eroarea 0 (minim absolut) pe care o \u0219i atinge uneori \u00eentr-un num\u0103r mic de itera\u021bii.<br \/>\nAr mai trebui justificat \u0219i de ce nu se mic\u0219oreaz\u0103 \u03c1 c\u00e2t de mult posibil, a\u0219a cum se face atunci c\u00e2nd se m\u0103re\u0219te, ci numai p\u00e2n\u0103 c\u00e2nd coboar\u0103 sub valoarea ini\u021bial\u0103 (din respectiva itera\u021bie) a erorii J. Nu caut s\u0103 minimizez eroarea prin c\u0103utarea unui pas optim la fiecare itera\u021bie (ca la metodele de tip Newton-Raphson sau tip cvasi-Newton <strong>[1]<\/strong>) pentru c\u0103 \u00een acest fel am cobor\u00ee foarte rapid (eventual \u00eentr-o singur\u0103 itera\u021bie) \u00een cel mai apropiat minim <strong>local<\/strong>. Pentru evitarea minimelor locale prefer\u0103m o cobor\u00e2re mai \u00eenceat\u0103 \u0219i, \u00een orice caz, prin pa\u0219i c\u00e2t mai mari posibili. Alegerea de pa\u0219i mari (\u03c1 mare) implic\u0103 at\u00e2t posibilitatea de a \u201ds\u0103ri\u201d \u00een alte regiuni de minim, c\u00e2t \u0219i salturi de o parte pe alta \u201da gropii\u201d (a \u201dv\u0103ii\u201d) \u00een care se g\u0103se\u0219te minimul local, ceea ce conduce la modificarea direc\u021biei gradientului de la o itera\u021bie la alta. \u00cen acest fel, pe mai mute itera\u021bii se efectueaz\u0103 \u0219i o c\u0103utare, cu pa\u0219i mari, \u00een direc\u021bii diferite.<br \/>\n\u00cen figurile 8, 9, 10, 11, 12 \u0219i 13 am reprezentat rezultatele simul\u0103rilor pentru fiecare din cele 6 probleme \u00een parte.<br \/>\nCurbele 1 reprezint\u0103 metoda clasic\u0103 <strong>BP<\/strong>, curbele 2 metoda nou\u0103 propus\u0103 aici, iar curbele 3 metoda <strong>CGBP<\/strong> considerat\u0103 ca cea mai bun\u0103 dintre metodele testate anterior.<br \/>\nSuperioritatea noii metode nu mai are nevoie de argumente. Este de remarcat totu\u0219i faptul c\u0103, chiar dac\u0103 maximul erorii din metoda NBP este uneori mai mare dec\u00e2t minimul erorii din celelalte metode, pentru fiecare set de ponderi ini\u021biale eroarea NBP a fost cea mai mic\u0103 fa\u021b\u0103 de eroarea la celelalte metode. Cu alte cuvinte, atunci c\u00e2nd NBP \u201dmerge r\u0103u\u201d, celelalte metode \u201dmerg \u0219i mai r\u0103u\u201d.<br \/>\nDe asemenea, utilizatorul nu este \u00eenc\u0103rcat cu nicio sarcin\u0103 de alegerea lui \u03c1 sau, cum e \u00een cazul CGBP (cea mai bun\u0103 metod\u0103 clasic\u0103), \u0219i a lui I. Comportarea metodelor clasice este extrem de puternic influen\u021bat\u0103 de alegerea acestor parametrii. La noua metod\u0103 nu mai este necesar\u0103 nicio selec\u021bie preliminar\u0103 a utilizatorului.<\/p>\n<p>5. <span style=\"text-decoration: underline;\">Concluzii<\/span><br \/>\nMetodele obi\u0219nuite de tip BP necesit\u0103 efectuarea de test\u0103ri pentru a determina valoarea optim\u0103 a lui \u03c1, valoare ce depinde de problema dat\u0103 spre \u00eenv\u0103\u021bare. Metodele mai rafinate necesit\u0103 \u0219i determinarea lui I.<br \/>\nA\u0219adar, pe l\u00e2ng\u0103 convergen\u021ba mai rapid\u0103 a metodei propuse, aceasta are \u0219i avantajul de a scuti utilizatorul de a efectua \u00eencerc\u0103ri preliminare pentru a determina pe \u03c1 \u0219i pe I. Trebuie remarcat faptul c\u0103 schimb\u0103rile efectuate \u00een programele de simulare la trecerea de la o metod\u0103 la alta au fost c\u00e2t mai mici posibil, c\u0103ut\u00e2ndu-se s\u0103 se modifice timpii de rulare numai pe baza cre\u0219terii complexit\u0103\u021bii calculelor.<br \/>\nCu toate c\u0103 re\u021belele neurale sunt structuri tipic paralele, simularea s-a realizat pe un calculator \u0219i cu un algoritm secven\u021bial. \u00cen cazul implement\u0103rii unei structuri paralele, compara\u021bia ar trebui f\u0103cut\u0103 nu la acela\u0219i timp de rulare, ci la acela\u0219i num\u0103r de itera\u021bii. \u00centr-un asemenea caz, superioritatea metodei propuse <strong>NBP<\/strong> s-ar accentua.<\/p>\n<p>6. <span style=\"text-decoration: underline;\">Bibliografie<\/span><\/p>\n<ol>\n<li>Battiti, R. ; Masulli, F. \u2013 \u201dBFGS Optimization for faster and automated supervised learning\u201d \u2013 \u00een \u201cProc. of the ICANN\u201d \u2013 Espoo, Finlanda, iunie, 1991.<\/li>\n<li>Benaim, M.; Tomasini, L. \u2013 \u201cCompetitive and self-organizing algorithms based on the minimisation of an information criteria\u201d \u2013 \u00een \u201cArtificial Neural Networks&#8221;\/ Kohonen, T.; Mkisara, K. ; Simula, O. ; Kanga, J.(Ed.) \u2013 Elsevier, Olanda, 1991.<\/li>\n<li>Hornik, K. ; Stinchcombe, M. ; White, H. \u2013 \u201cMultiplayer feedforward networks are universal approximators\u201d \u2013 Neural Networks, nr.2 \u2013 1989; pag. 359 \u2013 366.<\/li>\n<li>Hou, T. \u2013 H.; Lin, L. \u2013 \u201cManufacturing process monitoring using neural networks&#8221; &#8211; Computers &amp; Elect. Engineering \u2013 Vol. 19, nr. 2, 1993 \u2013 pag. 129 \u2013 141.<\/li>\n<li>Hush, D.; Abdallah, C.; Horne, B. \u2013 \u201cThe recursive neural network and its applications in control theory\u201d \u2013 Computer &amp; Elect. Engineering \u2013 Vol. 19, nr. 4, 1993 \u2013pag. 333-341.<\/li>\n<li>Jacobs, R.A. \u2013 \u201cIncreased rates of convergence through learning rate adaptation\u201d \u2013 Neural Networks \u2013 vol. 1, 1988 \u2013 pag. 295-307.<\/li>\n<li>Krse, B.J.A.; Smagt, P.P. van der \u2013 \u201cAn Introduction to Neural Networks\u201d &#8211; Universitatea din Amsterdam, 1993.<\/li>\n<li>Narendra, K.; Parthasarathy, K. \u2013 \u201cIdentification and control of dynamical systems using neural networks\u201d \u2013 IEEE Trans. On Neural Networks \u2013 Vol.1, nr. 1, 1990 \u2013 pag. 4-27.<\/li>\n<li>Ooyen, A. van; Nienhuis, B. \u2013 \u201cImproving the convergence of the back-propagation algorithm\u201d \u2013 Neural Networks \u2013vol.5, 1992 \u2013 pag.465-471.<\/li>\n<li>Philips, S.; Wiles, J. \u2013 \u201cExponential generalization from a polynomial number of examples in a combinatorial domain\u201d \u2013 \u00een \u201cProceedings of the International Joint Conference on Neural Networks: IJCNN\u201993\u201d- Nagoya, Japonia, octombrie, 1993, pag. 505-508.<\/li>\n<li>Pineda, F. J. \u2013 \u201cGeneralization of backpropagation t recurrent and higher order neural networks\u201d \u2013 \u00een \u201cProceedings of IEEE Conference on Neural Information Processing Systems\u201d\/Anderson, D.Z. (Ed.) \u2013 Denver, CO \u2013 noiembrie, 1987.<\/li>\n<li>Pineda, F.J. \u2013 \u201cDynamics and architecture for neural computation\u201d \u2013 Journal of Complexity \u2013 nr. 4, 1988 \u2013 pag. 216-245.<\/li>\n<li>Pineda, F.J. \u2013 \u201cRecurrent backpropagation and the dynamical approach to adaptive neural computation\u201d \u2013 Neural Computation \u2013 nr.1, 1989 \u2013 pag.161-172.<\/li>\n<li>Pourboghrat, F. \u2013 \u201cAdaptive neural controller design for robots\u201d \u2013 Computers &amp; Elect. Engineering \u2013 Vol.19, nr. 4, 1993 \u2013 pag. 277-288.<\/li>\n<li>Ribar, S.; Koruga, D. \u2013 \u201cNeural networks controller simulation\u201d \u2013 \u00een \u201cProc. of the ICANN\u201d \u2013 Espoo, Finlanda, iunie, 1991.<\/li>\n<li>Silva, F.M.; Almeida, L.B. \u2013 \u201cSpeeding up backpropagation\u201d \u2013 \u00een \u201cAdvanced neural computers\u201d\/Eckmiller, R (Ed.) \u2013 Olanda, 1990 \u2013 pag. 151-160.<\/li>\n<li>Volovici, D. \u2013 \u201cFiabilitatea proceselor tehnologice flexibile\u201d (Tez\u0103 Doctorat) \u2013 Universitatea Politehnica Bucure\u0219ti, Facultatea de Electronic\u0103 \u0219i Telecomunica\u021bii, 1994.<\/li>\n<li>Williams, R.J.; Zipser, D. \u2013 \u201cA learning algorithm for continually running fully recurrent neural networks\u201d \u2013 Neural Computation \u2013 nr.1; 1989, MIT \u2013 pag.270-280.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>COMPARA\u021aIE \u00ceNTRE METODE DE \u00ceNV\u0102\u021aARE DE TIP BACKPROPAGATION Abstract: Acest articol analizeaz\u0103 \u0219i compar\u0103 diverse \u00eembun\u0103t\u0103\u021biri aduse metodei backpropagation de ajustare a ponderilor unei re\u021bele feed-forward. Pentru aceasta se simuleaz\u0103 comportarea re\u021belelor pentru 6 aplica\u021bii tipice diverse. \u00cen plus, se &hellip; <a href=\"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/?page_id=410\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=\/wp\/v2\/pages\/410"}],"collection":[{"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=410"}],"version-history":[{"count":30,"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=\/wp\/v2\/pages\/410\/revisions"}],"predecessor-version":[{"id":437,"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=\/wp\/v2\/pages\/410\/revisions\/437"}],"wp:attachment":[{"href":"https:\/\/web.ulbsibiu.ro\/daniel.volovici\/html\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=410"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}