<div dir="ltr"><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;color:#000000">Hello Everyone, </div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;color:#000000"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;color:#000000">Last year I presented the Hrrformer paper in the lab. It's been accepted by ICML 23 and finally, the paper and code are released. Hrrformer is a neuro-symbolic self-attention model with linear 𝒪(n) time and space complexity. 23× faster and consumes 24× less memory than Transformer. SOTA performance for even over sequence length n≥100,000. Able to learn with a single layer and converges 10× faster in LRA benchmark. The code is written in Jax, however, it is fairly easy to implement it in PyTorch. If you are using self-attention and are interested to reduce the time and space complexity of your model, you can give it a try. </div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;color:#000000"><br></div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;color:#000000">Paper: <a href="https://arxiv.org/pdf/2305.19534.pdf">https://arxiv.org/pdf/2305.19534.pdf</a> </div><div class="gmail_default" style="font-family:tahoma,sans-serif;font-size:small;color:#000000">Code: <a href="https://github.com/NeuromorphicComputationResearchProgram/Hrrformer">https://github.com/NeuromorphicComputationResearchProgram/Hrrformer</a> </div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><span><div dir="ltr" style="margin-left:0pt" align="left"><font face="tahoma, sans-serif"><br><table style="border:none;border-collapse:collapse"><colgroup><col width="78"><col width="38"><col width="38"><col width="38"><col width="38"><col width="38"><col width="38"><col width="38"><col width="38"><col width="239"></colgroup><tbody><tr style="height:49.99999999999999pt"><td rowspan="2" style="vertical-align:top;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><span style="border:none;display:inline-block;overflow:hidden;width:62px;height:96px"><img alt="Logo

Description automatically generated" src="https://lh5.googleusercontent.com/EDkPwNa5Jo6__lxjUr3VVP1HFKD6kZfw1kRa_GKZDYznNWauxirGJJQX7_NCJmuLM77Ub-Lc68fVgCRG9IBNGevBv_dsl-9wCreQsTMha_oMY9yVCVEVqGDjABpnYBwkPcVALfBIpcdAl5T_ovdYCitJV9kL98LxQL_soOXK2nPkQ6v4mc01URdj71imd-dOT7rTCw" width="62" height="96" style="margin-left:0px;margin-top:0px"></span></span></p></td><td colspan="9" style="vertical-align:top;overflow:hidden"><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><font face="tahoma, sans-serif">Regards,</font></span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><font face="tahoma, sans-serif">Mohammad Mahmudul Alam</font></span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="color:rgb(0,0,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><font face="tahoma, sans-serif">Ph.D. candidate, Computer Science, UMBC</font></span></p></td></tr><tr style="height:5.892578125000007pt"><td style="vertical-align:top;overflow:hidden"><p dir="ltr" style="line-height:1.2;text-align:center;margin-top:0pt;margin-bottom:0pt"><a href="https://mahmudulalam.github.io" target="_blank"><span style="color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><span style="border:none;display:inline-block;overflow:hidden;width:29px;height:29px"><img src="https://lh3.googleusercontent.com/5aCzE7t4DSxVcfx2SfDozY5M3OwdUt7k795SLZG6uQdZW_UKA-uCJFAulGcXipTqZFR2vBVhN02S75b_aA20-gQOelBWkhMs_SsLnRPD0Yzw5mkX1NdxKqYJON7nNWlz8Di-ymcITrr5ggxf7IOeuBSxMk_jIC7fk0vP7Pt6yK2NcHvA_ajZmOudWYFVeIRL-b_SZQ" width="29" height="29" style="margin-left:0px;margin-top:0px"></span></span></a></p></td><td style="vertical-align:top;overflow:hidden"><p dir="ltr" style="line-height:1.2;text-align:center;margin-top:0pt;margin-bottom:0pt"><a href="https://github.com/MahmudulAlam" target="_blank"><span style="color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><span style="border:none;display:inline-block;overflow:hidden;width:29px;height:29px"><img src="https://lh4.googleusercontent.com/SdyILHRrk7w84zgaNkoFdSPXT7MNihvR_nJDoYrJs7U856QQGyrpxYPcS54YH0mmN-7a60cGMFO1ltYseX4FH4Rb7BWLMA-QgMKP1ZgviBRDB-3hqITiKA6ncdptxHj92Wt6_XHztR9DW3xPGFCH0vTrR5QKvcCwzENFHxYK49bmGb9vldYdssV_mkT9fXrLK6co3Q" width="29" height="29" style="margin-left:0px;margin-top:0px"></span></span></a></p></td><td style="vertical-align:top;overflow:hidden"><p dir="ltr" style="line-height:1.2;text-align:center;margin-top:0pt;margin-bottom:0pt"><a href="https://www.linkedin.com/in/mahmudul-alam/" target="_blank"><span style="color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><span style="border:none;display:inline-block;overflow:hidden;width:29px;height:29px"><img src="https://lh6.googleusercontent.com/hMvD4hfaZ3QptmxOHBeS9okgNX2qpyB5JZ1qRyVO3TGg5L6Fsn54LIjrU3qXP1Fv53paeku0T_JmH7B8-NgMqhHPIR0N6ZGijtKuowXaIopfz5hvfFEKcX9S8ZCcHWin5jGh5vbeHkYI4JC5y2rDuwEURTkja4OdO3o8O6B0I03k6IfIcJanjBe3P576haqEg-XiBg" width="29" height="29" style="margin-left:0px;margin-top:0px"></span></span></a></p></td><td style="vertical-align:top;overflow:hidden"><p dir="ltr" style="line-height:1.2;text-align:center;margin-top:0pt;margin-bottom:0pt"><a href="https://scholar.google.com/citations?view_op=list_works&hl=en&user=9z9HFSEAAAAJ" target="_blank"><span style="color:rgb(17,85,204);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><span style="border:none;display:inline-block;overflow:hidden;width:34px;height:34px"><img src="https://lh5.googleusercontent.com/06HpWYLVRF8Dlhx2vSZaXR-mWHD06v3twD-aPoW4bXokDftXJtX7PMY52wYmBwIxZfhOHeMe6v4mk3Etj6coXDn8bOdJbzH5MCRvnmYL9gJU4SNg1SrEBUnCil_G-bLLizuu4aT1IXRuGNv9alcwOLB2FhYarOh2kfs4SJHIkAox_1mzf26MfiCa4nbZKxqqroTYyQ" width="34" height="34" style="margin-left:0px;margin-top:0px"></span></span></a></p></td><td style="vertical-align:top;overflow:hidden"><br></td><td style="vertical-align:top;overflow:hidden"><br></td><td style="vertical-align:top;overflow:hidden"><br></td><td style="vertical-align:top;overflow:hidden"><br></td><td style="vertical-align:top;overflow:hidden"><br></td></tr></tbody></table></font></div></span></div></div></div></div>