Skip to content

The job command

The Code Behind The Command

The core functionality and implementation behind the /ljobs command is handled by the job_posts modules.

When the /ljobs command is called the job_post_factory.py module calls the job_scrapper.py module to send a request to LinkedIn jobs with the provided search parameters (job title, location) or the default search parameters BI Data Analyst, Egypt. if none was provided.

Then the returned data is passed to the TgJobPost data class in the job_post_creator.py module to further parse and format the job data to make it ready for being sent in telegram chat using the send_job_posts function in the job_post_sender.py module.

The job_post_factory.py module

Click to view the job_post_factory.py module
channel_jobs_updater(bot: TeleBot) -> None

summary : This function updates the job posting in the channel using the CHANNEL_ID variable from .env file

Parameters:

Name Type Description Default
bot TeleBot

description : bot instance

 required
Source code in src/job_posts/job_post_factory.py 
100
101
102
103
104
105
106
107
108
109
110
111
def channel_jobs_updater(bot: TeleBot) -> None:
    """_summary_ : This function updates the job posting in the channel using the CHANNEL_ID variable from .env file

    Parameters
    ----------
    bot : TeleBot
        _description_ : bot instance
    """
    # Creating jobs using the jobs factory
    jobs = jobs_factory()
    # Sending jobs to the channel
    send_job_posts(jobs, bot, channel_id=CHANNEL_ID)
job_scrapper(scrapper: LinkedinScrapper, search_params: tuple[str, str] = None) -> list[dict]

summary : This function creates the linkedin scrapper object and retrieves the formatted data.

Parameters:

Name Type Description Default
scrapper LinkedinScrapper

description : The linkedin scrapper object.

 required

Returns:

Type Description
list[dict]

description : A list of dicts containing the scrapped jobs data for linkedin.
Source code in src/job_posts/job_post_factory.py 
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
def job_scrapper(
    scrapper: LinkedinScrapper, search_params: tuple[str, str] = None
) -> list[dict]:
    """_summary_ : This function creates the linkedin scrapper object and retrieves the formatted data.

    Parameters
    ----------
    scrapper : LinkedinScrapper
        _description_ : The linkedin scrapper object.

    Returns
    -------
    list[dict]
        _description_ : A list of dicts containing the scrapped jobs data for linkedin.
    """
    # Creating the scrapper object.
    scrapper = scrapper()

    # If search parameters were provided unpack them and pass them to the scrapper object.
    if search_params:
        # Setting up the search parameters tuple(job title, location).
        scrapper.set_search_params(*search_params)

    # Starting the scrapping process
    scrapper.scrape_jobs()

    # Retuning the formatted_data attribute contain the scraped formatted data from linkedin.
    return scrapper.formatted_data
jobs_factory(search_params: tuple[str, str] = None) -> list[dict]

summary : This function creates the scrapper object and the post creator objects.

Returns:

Type Description
list[dict]

description : A list of dict containing the formatted posts ready to send to telegram chat.
Source code in src/job_posts/job_post_factory.py 
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
def jobs_factory(search_params: tuple[str, str] = None) -> list[dict]:
    """_summary_ : This function creates the scrapper object and the post creator objects.

    Returns
    -------
    list[dict]
        _description_ : A list of dict containing the formatted posts ready to send to telegram chat.
    """
    # If search parameters were passed as an argument pass them into the scrapper.
    if search_params:
        # Passing the scrapper to the job_scrapper function with the search parameters if provided.
        jobs = job_scrapper(scrapper=LinkedinScrapper, search_params=search_params)
    else:
        # If no search parameters were provided, only pass the scrapper (will scrap for the default values)
        jobs = job_scrapper(scrapper=LinkedinScrapper)

    # Returning the created posts.
    return post_creator(data=jobs, creator=TgJobPost)
post_creator(data: list[dict], creator: TgJobPost) -> list[dict]

summary : This function creates the telegram job post creator objects to create job posts for the telegram channel.

Parameters:

Name Type Description Default
data list[dict]

description : A list of dict containing the jobs data.

 required
creator TgJobPost

description : The telegram job posts creator object.

 required

Returns:

Type Description
list[dict]

description: A list of dict containing the formatted posts ready to send to telegram chat.
Source code in src/job_posts/job_post_factory.py 
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
def post_creator(data: list[dict], creator: TgJobPost) -> list[dict]:
    """_summary_ : This function creates the telegram job post creator objects to create job posts for the telegram channel.

    Parameters
    ----------
    data : list[dict]
        _description_ : A list of dict containing the jobs data.
    creator : TgJobPost
        _description_ : The telegram job posts creator object.

    Returns
    -------
    list[dict]
        _description_: A list of dict containing the formatted posts ready to send to telegram chat.
    """

    # Creating the telegram job post creator object
    creator = creator()

    # Creating the job post using the 'create_post()' method
    creator.create_posts(data)

    # Retuning the created posts stored in the 'TgJobPost.posts' attribute
    return creator.posts

The job_post_factory is the factory that handles creating the job posts process, from scrapping and parsing the data, formatting it for telegram messages, to sending it in chat.

How does it work

  1. It calls the scrapper class LinkedinScrapper from the job_scrapper.py module.
  2. Passes the returned data from the LinkedinScrapper class to the TgJobPost class in the job_post_creator.py module to extract jobs information and format it for telegram.
  3. It uses the send_job_posts function from the job_post_sender.py module to send each created post as a separate message in the telegram chat.

The job_scrapper.py module

The job_scrapper module contains the LinkedinScrapper data class which scrapes and collects job posts, parses, and formats them.

The LinkedinScrapper uses the requests library to send the request to LinkedIn, and BeautifulSoap to parse the HTML response.

Click to view the LinkedinScrapper data class

Bases: Scrapper

summary : This data class scraps linkedin for the specified search key word and location in the .ev file.

Source code in src/job_posts/job_scrapper.py 
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
@dataclass(slots=True)
class LinkedinScrapper(Scrapper):
    """_summary_ : This data class scraps linkedin for the specified search key word and location in the .ev file."""

    _job_tile: str = "BI Data Analyst"

    _location: str = "Egypt"

    # The url to send request to and get data back.
    url: str = "https://www.linkedin.com/jobs/search?keywords={JOB_TITLE}&location={LOCATION}&f_TPR=r604800"

    # Default list to hold raw html data.
    raw_data: list[dict] = field(default_factory=list)

    # Default list to hold parsed data.
    parsed_data: list[tuple[str, str, str, str]] = field(default_factory=list)

    # Default list to hold final formatted data ready for use.
    formatted_data: list[dict] = field(default_factory=list)

    def set_search_params(self, job_tile: str, location: str) -> None:
        """_summary_ :  This method sets the search parameters for the linkedin jobs.

        Parameters
        ----------
        job_tile : str
            _description_ : The jobs title to search linkedin jobs for.
        location : str
            _description_ : The location to search for jobs in.
        """
        # Setting the job title instance variable.
        self._job_tile = job_tile
        # Setting the location instance variable.
        self._location = location

    def scrape_jobs(self) -> None:
        """This method start the scrapping process."""

        # Formatting the url with the job title and location.
        self.url = self.url.format(JOB_TITLE=self._job_tile, LOCATION=self._location)

        # Collecting the data.
        self.collect_data()
        # Parsing the data.
        self.parse_data()
        # Formatting the data.
        self.format_data()

    def collect_data(self):
        """This Method sends calls the url using the request lib and gets back the data from linkedin"""
        # Getting the response from the website.
        response = requests.get(self.url)

        # Parsing the response.
        soup = BeautifulSoup(response.content, "html.parser")

        # Getting the job cards.
        html_data = soup.find_all(
            "div",
            class_="base-card relative w-full hover:no-underline focus:no-underline base-card--link base-search-card base-search-card--link job-search-card",
        )

        # Returning the raw collected html data.
        self.raw_data = html_data

    def parse_data(self):
        """This Method parses data and extracts the job's details."""

        # Getting the data out of the instance variable for clarity.
        data = self.raw_data

        # Looping over the raw html page data and extracting jobs details.
        for job in data:
            # Getting the job title.
            job_title = job.find("h3", class_="base-search-card__title").text.strip()

            # Getting the company name.
            job_company = job.find(
                "h4", class_="base-search-card__subtitle"
            ).text.strip()

            # Getting the job location.
            job_location = job.find(
                "span", class_="job-search-card__location"
            ).text.strip()

            # Getting the job link.
            apply_link = job.find("a", class_="base-card__full-link")["href"]

            # Appending the job details to class variable list as a tuple.
            self.parsed_data.append((job_title, job_company, job_location, apply_link))

    def format_data(self):
        """This Method formats data after being parsed into a desired format"""

        # Getting the data out of the instance variable for clarity.
        data = self.parsed_data

        # Looping over the parsed data and formatting it, to be used by the TgJobPost class to create jobs posting posts for telegram.
        for job in data:
            # For each job in the data create a dict object that holds each job detail in a separate key.
            job_details = {
                # Getting the job title from the first item in the tuple.
                "job_title": job[0],
                # Getting the job company from the second item in the tuple
                "job_company": job[1],
                # Getting the job location from the third item in the tuple
                "job_location": job[2],
                # Getting the job link from the forth item in the tuple
                "apply_link": job[3],
            }
            # Adding this dict to the formatted_data instance variable.
            self.formatted_data.append(job_details)
collect_data()

This Method sends calls the url using the request lib and gets back the data from linkedin

Source code in src/job_posts/job_scrapper.py 
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
def collect_data(self):
    """This Method sends calls the url using the request lib and gets back the data from linkedin"""
    # Getting the response from the website.
    response = requests.get(self.url)

    # Parsing the response.
    soup = BeautifulSoup(response.content, "html.parser")

    # Getting the job cards.
    html_data = soup.find_all(
        "div",
        class_="base-card relative w-full hover:no-underline focus:no-underline base-card--link base-search-card base-search-card--link job-search-card",
    )

    # Returning the raw collected html data.
    self.raw_data = html_data
format_data()

This Method formats data after being parsed into a desired format

Source code in src/job_posts/job_scrapper.py 
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
def format_data(self):
    """This Method formats data after being parsed into a desired format"""

    # Getting the data out of the instance variable for clarity.
    data = self.parsed_data

    # Looping over the parsed data and formatting it, to be used by the TgJobPost class to create jobs posting posts for telegram.
    for job in data:
        # For each job in the data create a dict object that holds each job detail in a separate key.
        job_details = {
            # Getting the job title from the first item in the tuple.
            "job_title": job[0],
            # Getting the job company from the second item in the tuple
            "job_company": job[1],
            # Getting the job location from the third item in the tuple
            "job_location": job[2],
            # Getting the job link from the forth item in the tuple
            "apply_link": job[3],
        }
        # Adding this dict to the formatted_data instance variable.
        self.formatted_data.append(job_details)
parse_data()

This Method parses data and extracts the job's details.

Source code in src/job_posts/job_scrapper.py 
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
def parse_data(self):
    """This Method parses data and extracts the job's details."""

    # Getting the data out of the instance variable for clarity.
    data = self.raw_data

    # Looping over the raw html page data and extracting jobs details.
    for job in data:
        # Getting the job title.
        job_title = job.find("h3", class_="base-search-card__title").text.strip()

        # Getting the company name.
        job_company = job.find(
            "h4", class_="base-search-card__subtitle"
        ).text.strip()

        # Getting the job location.
        job_location = job.find(
            "span", class_="job-search-card__location"
        ).text.strip()

        # Getting the job link.
        apply_link = job.find("a", class_="base-card__full-link")["href"]

        # Appending the job details to class variable list as a tuple.
        self.parsed_data.append((job_title, job_company, job_location, apply_link))
scrape_jobs() -> None

This method start the scrapping process.

Source code in src/job_posts/job_scrapper.py 
71
72
73
74
75
76
77
78
79
80
81
82
def scrape_jobs(self) -> None:
    """This method start the scrapping process."""

    # Formatting the url with the job title and location.
    self.url = self.url.format(JOB_TITLE=self._job_tile, LOCATION=self._location)

    # Collecting the data.
    self.collect_data()
    # Parsing the data.
    self.parse_data()
    # Formatting the data.
    self.format_data()
set_search_params(job_tile: str, location: str) -> None

summary : This method sets the search parameters for the linkedin jobs.

Parameters:

Name Type Description Default
job_tile str

description : The jobs title to search linkedin jobs for.

 required
location str

description : The location to search for jobs in.

 required
Source code in src/job_posts/job_scrapper.py 
56
57
58
59
60
61
62
63
64
65
66
67
68
69
def set_search_params(self, job_tile: str, location: str) -> None:
    """_summary_ :  This method sets the search parameters for the linkedin jobs.

    Parameters
    ----------
    job_tile : str
        _description_ : The jobs title to search linkedin jobs for.
    location : str
        _description_ : The location to search for jobs in.
    """
    # Setting the job title instance variable.
    self._job_tile = job_tile
    # Setting the location instance variable.
    self._location = location

The job_post_creator.py module

The job_post_creator.py module contains the TgJobPost data class which takes in a dictionary of lists (created by the LinkedinScrapper data class) containing the job title, company, location, and job post link.

It creates a dictionary of two key:value paris for each job post as the following: {job_details:str, job_link: str}.

job_post_creator.py
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
# looping over the jobs data list[dict] and creating a [dict] for each job containing the 'job details' and the 'job links'.
for job in jobs_data:

    # This is a dict that will hold the job details and the job links for each job.
    post = {
        # This key hold the job details formatted for posting using the post_template.
        "job_details": self.post_template.format(
            job["job_title"],
            job["job_company"],
            job["job_location"],
            # Grab the first apply link for and embed it inside 'APPLY HERE'
            ## Using Markdown syntax => [Text](Link to embed).
            f"[ LINK 🔗]({job['apply_link']})",
        ),
        # This key hold the list of apply links for this job.
        "job_link": job["apply_link"],
    }

• The job_details: contains the job information as a string.

• The job_link: contains the job post URL on linkedin.

Click to view the TgJobPost data class

Bases: TgPost

summary : This data class creates posts from the scrapped data.

Source code in src/job_posts/job_post_creator.py 
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
@dataclass(slots=True)
class TgJobPost(TgPost):
    """_summary_ : This data class creates posts from the scrapped data."""

    # This is the template that will be used to create the post.
    post_template: str = (
        "• *Job Title* : {} \n"
        "• *Company* : {} \n"
        "• *Location* : {} \n"
        "• *Job Link* : {} \n"
    )

    # This is a list that will hold the final posts.
    posts: list = field(default_factory=list)

    def create_posts(self, jobs_data: list[dict]) -> list[dict]:
        """_summary_ : This method loop over the jobs_data list and create a post for each job.

        Parameters
        ----------
        jobs_data : list[dict]
            _description_ : This is the list of jobs data that will be used to create the posts.

        Returns
        -------
        list[dict]
            _description_ : A list of dicts that contain 'job_details' and 'job_links' for each jobs.

        """
        # looping over the jobs data list[dict] and creating a [dict] for each job containing the 'job details' and the 'job links'.
        for job in jobs_data:

            # This is a dict that will hold the job details and the job links for each job.
            post = {
                # This key hold the job details formatted for posting using the post_template.
                "job_details": self.post_template.format(
                    job["job_title"],
                    job["job_company"],
                    job["job_location"],
                    # Grab the first apply link for and embed it inside 'APPLY HERE'
                    ## Using Markdown syntax => [Text](Link to embed).
                    f"[ LINK 🔗]({job['apply_link']})",
                ),
                # This key hold the list of apply links for this job.
                "job_link": job["apply_link"],
            }

            # Adding the formatted job into the 'self.posts' list of the class.
            self.posts.append(post)
create_posts(jobs_data: list[dict]) -> list[dict]

summary : This method loop over the jobs_data list and create a post for each job.

Parameters:

Name Type Description Default
jobs_data list[dict]

description : This is the list of jobs data that will be used to create the posts.

 required

Returns:

Type Description
list[dict]

description : A list of dicts that contain 'job_details' and 'job_links' for each jobs.
Source code in src/job_posts/job_post_creator.py 
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def create_posts(self, jobs_data: list[dict]) -> list[dict]:
    """_summary_ : This method loop over the jobs_data list and create a post for each job.

    Parameters
    ----------
    jobs_data : list[dict]
        _description_ : This is the list of jobs data that will be used to create the posts.

    Returns
    -------
    list[dict]
        _description_ : A list of dicts that contain 'job_details' and 'job_links' for each jobs.

    """
    # looping over the jobs data list[dict] and creating a [dict] for each job containing the 'job details' and the 'job links'.
    for job in jobs_data:

        # This is a dict that will hold the job details and the job links for each job.
        post = {
            # This key hold the job details formatted for posting using the post_template.
            "job_details": self.post_template.format(
                job["job_title"],
                job["job_company"],
                job["job_location"],
                # Grab the first apply link for and embed it inside 'APPLY HERE'
                ## Using Markdown syntax => [Text](Link to embed).
                f"[ LINK 🔗]({job['apply_link']})",
            ),
            # This key hold the list of apply links for this job.
            "job_link": job["apply_link"],
        }

        # Adding the formatted job into the 'self.posts' list of the class.
        self.posts.append(post)

The job_post_sender.py module

This module has only one simple function, send_job_posts which takes in the dictionary of list containing the job details and job link key, value pairs (Created by the TgJobPost data class) and sends each one in a separate message, with the job_details as the message body, and the job_link as an inline keyboard button directing to the job apply link.

Click to view the send_job_posts function

summary : Loops over the provided job post list and send each post in a separate message.

Parameters:

Name Type Description Default
posts list[dict]

description : The list of job posts created by the telegram post creator.

 required
bot TeleBot

description

 required
msg Message

description : The Message Object

 None
channel_id str

description, by default None : The channel id.

 None
Source code in src/job_posts/job_post_sender.py 
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def send_job_posts(
    posts: list[dict], bot: TeleBot, msg: Message = None, channel_id: str = None
) -> None:
    """_summary_ : Loops over the provided job post list and send each post in a separate message.

    Parameters
    ----------
    posts : list[dict]
        _description_ : The list of job posts created by the telegram post creator.
    bot : TeleBot
        _description_
    msg : Message
        _description_ : The Message Object
    channel_id : str, optional
        _description_, by default None : The channel id.
    """

    # Looping over the posts list and sending each post to the user.
    for post in posts:
        bot.send_message(
            chat_id=channel_id or msg.chat.id,
            text=post["job_details"],
            reply_markup=jobs_post_inline_kb(post["job_link"]),
            parse_mode="Markdown",
        )

The send_job_posts function uses the jobs_post_inline_kb function from the keyboards module folder to create the inline keyboard.

Click to view the jobs_post_inline_kb function

summary : This function creates the inline keyboard markup for the job links.

Parameters:

Name Type Description Default
job_link str

description : job link URL.

 required

Returns:

Type Description
InlineKeyboardMarkup

description : The inline keyboard markup for the job links.
Source code in src/tgbot/keyboards/inline/inline_keyboards.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
def jobs_post_inline_kb(job_link: str) -> InlineKeyboardMarkup:
    """_summary_ : This function creates the inline keyboard markup for the job links.

    Parameters
    ----------
    job_link : str
        _description_ : job link URL.

    Returns
    -------
    InlineKeyboardMarkup
        _description_ : The inline keyboard markup for the job links.
    """

    # Creating the inline keyboard markup instance, for the job links.
    inline_kb = InlineKeyboardMarkup()

    # Setting the keyboard row width to 1; so the links will be displayed in one row.
    inline_kb.row_width = 1

    inline_kb.add(
        InlineKeyboardButton("👆 Click Here To Apply 👆", url=job_link),
    )

    # Returning the inline keyboard markup.
    return inline_kb

Using The Command

The ljobs command can be used alone or with arguments (search parameters) to scrap LinkedIn jobs for last week posted jobs returning each job in a separate message with the job information and the job link for applying.

If the /ljobs command was sent in chat without any arguments or search parameters it will default to searching for BI Data Analyst Roles in Egypt like in the following GIF.

ljob_command
/ljobs command.

Searching With Arguments

The /ljob command also supports custom job search by providing search parameters after the command in chat following the job title, location pattern | format as demonstrated in the following GIF.

ljob_command
/ljobs command with parameters.

Please note that

If you don't follow the job title, location pattern the bot will through an error telling you that this is an invalid search pattern, specifying the correct one to follow.

Search Format | Pattern

The /ljob command uses python's Regex library to verify the provided search parameters using param_validator function in the job_commands.py module.

The param_validator function does the validation using the following pattern.

param_validator function
# Compiling the command valid pattern.
pattern = re.compile(r"\D+,\D+")
Click to view the param_validator function

summary : This function takes in the job command parameter and validates it.

Parameters:

Name Type Description Default
parameter str

description : The parameter extracted from the message.

 required

Returns:

Type Description
bool

description : True if is valid, False it not.
Source code in src/tgbot/commands/job_commands.py 
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
def param_validator(parameter: str) -> bool:
    """_summary_ : This function takes in the job command parameter and validates it.

    Parameters
    ----------
    parameter : str
        _description_ : The parameter extracted from the message.

    Returns
    -------
    bool
        _description_ : True if is valid, False it not.
    """
    # Compiling the command valid pattern.
    pattern = re.compile(r"\D+,\D+")

    # lowering the param case and removing spaces.
    params = parameter.lower().replace(" ", "")

    # Checking if passed parameters is valid of not, and returning the result as a bool True | False.
    return bool(re.search(pattern, params))

Here's a live example of the bot's behavior when provided with invalid parameters.

ljob_command
/ljobs command pattern validator.