Project: INCC Crawler

Guilherme Latrova -

Source Code: https://github.com/guilatrova/incc-crawler

Stack: Python, Lambda, BeautifulSoup

In Brazil we have an index named INCC (Indice Nacional de Construção Civil) or Civil Construction National Index . Basically it tracks the inflation of construction material which may affect people that bough a house under construction.

I decided to create a crawler to notify me monthly through Telegram. Just because it seems fun :)

-

Highlights

If I had to name something that I enjoyed most on this project, I would mention how I applied chain of responsibility to build the Telegram message:

Chain of Responsibility
Chain of Responsibility is a behavioral design pattern that lets you pass requests along a chain of handlers. Upon receiving a request, each handler decides either to process the request or to pass it to the next handler in the chain.

Applied:

class MessageFactory:
    def build_message(self, overview: dtos.TableResult, complete: dtos.TableResult):
        builder = HeadlineBuilder()
        next_builder = builder.then(CurPeriodBuilder())
        next_builder = next_builder.then(PrevPeriodDiffBuilder())

        for prev_year in range(1, 5):
            next_builder = next_builder.then(PrevYearSamePeriodBuilder(prev_year))

        next_builder = next_builder.then(FooterDetailsBuilder())
        next_builder = next_builder.then(TrimmerBuilder())

        return builder.build(overview, complete, "")

It's interesting to avoid string concatenation mess. Feel free to see the implementation for yourself HERE.