[agents] Call for Shared Task (Multi-Topic Labelling Task) Participation at ICICS (IEEE Sponsored): Mowjaz Multi-Topic Labelling Task

[Gizem Varkonyi]

 [Apologies for multiple postings]
-------------------------------------------------------------------------
Mowjaz Multi-Topic Labelling Task
Website: https://www.just.edu.jo/icics/icics2021/mowjaz/

To be organized at ICICS 2021 (https://www.just.edu.jo/icics/icics2021/)
24 - 26 May
Virtual Conference
-------------------------------------------------------------------------------

--------------------------
Task Description:
--------------------------

Mowjaz (mowjaz.com) is an Arabic topical content aggregation mobile
application for news, sport, entertainment and other topics from top
publishers that users can follow. Mowjaz search engine and recommendation
system is built on top of NLP/NLU machine learning APIs that distinguish it
from any other Arabic content applications available, mainly focusing on
the users having the best experience and receiving content that is of their
interest. Mowjaz is a subsidiary of Mawdoo3.com, the world's biggest Arabic
website in terms of number of visitors.

One of Mowjaz's top AI powered models is Topic Multi-Labelling, which is
the focus of this shared task. This model is basically used to classify
articles based on their topics. Additionally, the model predicts multiple
topics in one article and is categorized to all possible topics that are
present within its content. Mowjaz's topics are classified into ten
categories and an article can be classified under as many topics as it
covers. This model helps users get and display the most relevant news to
their interests.  The enhanced user experience that Mowjaz offers makes one
news article be classified and shown under all the different topics that it
holds.

Participating systems are expected to select one or more of the ten topics
for each given article. They are evaluated based on their effectiveness and
efficiency.

Full task description can be found at:
https://www.just.edu.jo/icics/icics2021/mowjaz/

--------------------------
Dataset and Evaluation:
--------------------------
The dataset consists of 9,590 articles split into training, development and
testing sets.

The evaluation of the participating systems is done is two separate tracks
as follows:
- The first track is the effectiveness track, where the participating
systems are ranked based on their accuracy. Specifically, the F1 score is
used for this track. This track is mandatory for all participating systems.

- The second track, which is optional,  is the efficiency track.
Participating teams are asked to run their systems in a docker container
with memory constraints to measure their average testing times. A simple
tutorial of this process is provided in the GitHub repository of this task.
The score of each system is a weighted average of its F1 score and average
testing time (normalized) and the systems are ranked based on these scores.
Submissions for this track will be open in the last week of the competition
and results will be announced once after the end of the competition.

The second track is optional and it promotes the open-source mentality.
Thus, teams participating in it are asked to share any resources they use
such as external datasets (labeled or unlabeled), pre-trained models,
knowledge bases, etc.

------------
Timeline
------------

18-Feb-2021 Release of task website, dataset & Codalab competition
18-Mar-2021 Start of the evaluation phase for the effectiveness track
12-Apr-2021 Start of the evaluation phase for the efficiency track
19-Apr-2021 Deadline for submitting runs
20-Apr-2021 Results declared
23-Apr-2021 Single-Paragraph descriptions of participating systems due
03-May-2021 Working notes papers due
10-May-2021 Reviews on working notes papers due
17-May-2021 Final version of working notes papers due
24-May-2021 ICICS2021

----------------
Organizers
----------------
Mahmoud Al-Ayyoub, Department of Computer Science (CS) and Center of
Excellence for Innovative Projects (CEIP), Jordan University of Science and
Technology, Jordan
Haitham Selawi, Mawdoo3 Ltd, Jordan
Mohamed Zaghlol, Mawdoo3 Ltd, Jordan
Hussein Al-Natsheh, Mawdoo3 Ltd, Jordan
Samer Suileman, Center of Excellence for Innovative Projects (CEIP), Jordan
University of Science and Technology, Jordan
Ali Fadel, Department of Computer Science (CS), Jordan University of
Science and Technology, Jordan
Riham Badawi, Mawdoo3 Ltd, Jordan
Ahmed Morsy, Mawdoo3 Ltd, Jordan
Ibraheem Tuffaha, Mawdoo3 Ltd, Jordan
Mohannad Aljarrah, Center of Excellence for Innovative Projects (CEIP),
Jordan University of Science and Technology, Jordan

For regular updates subscribe to our mailing list:
mowjaz-task at googlegroups.com

For any question about this call, please send an email to Mahmoud Al-Ayyoub
:  maalshbool at just.edu.jo

Regards,
Organizers of the Mowjaz Multi-Topic Labelling Task
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cs.umbc.edu/pipermail/agents/attachments/20210302/f88f82f3/attachment-0001.html>