07/29/2021 | News release | Distributed by Public on 07/29/2021 19:10
Microservices: some people love them for how they can break monolithic applications into more manageable, distributed services; others hate them because of the complexity that can arise trying to coordinate dozens of distributed services just to accomplish a simple task. Wherever you may sit along the love/hate spectrum, there's no denying that distributed software architecture has been significantly impacted by the rise of microservices, which encourage reusability of components.
In this tutorial, I'll introduce the concept of microservices architecture and why Python is a great choice for creating microservices. But the best way to learn about microservices is to build one, so that's what we'll do:
Sound like fun? Let's get started.
To follow along with the code in this article, you can download and install our pre-built Spotify Videos environment, which contains a version of Python 3.9 and the packages used in this post.
In order to download this ready-to-use Python environment, you will need to create an ActiveState Platformaccount. Just use your GitHub credentials or your email address to register. Signing up is easy and it unlocks the ActiveState Platform's many benefits for you!
Or you could also use our State toolto install this runtime environment.
For Windows users, run the following at a CMD prompt to automatically download and install our CLI, the State Tool along with the Spotify Videosinto a virtual environment:
powershell -Command '& $([scriptblock]::Create((New-Object Net.WebClient).DownloadString('https://platform.activestate.com/dl/cli/install.ps1'))) -activate-default Pizza-Team/Spotify-Videos'
For Linux users, run the following to automatically download and install our CLI, the State Tool along with the Spotify Videosinto a virtual environment:
sh <(curl -q https://platform.activestate.com/dl/cli/install.sh) --activate-default Pizza-Team/Spotify-Videos
The evolution of microservices spans some three decades, starting with:
As with any buzzword, though, the term microservices has been used, misused, and abused. In general, a true microservice:
Python is a great choice for implementing microservices, not only because of the great benefits of the language itself, but also because it lends itself well to use in microservices architecture, via:
Spotify is a great way to get introduced to new songs, but I often wish I could check out the music video that goes with them. I can always plug the name of a song into YouTube and see if I get a hit, but that's more of a distraction than I'm usually willing to commit to. But what if I could get access to both the song and the video with a single click?
Imagine a microservice whose sole purpose is to find the most relevant YouTube video for the top 50 songs on the global playlist in Spotify. This microservice will:
The following diagram shows the system context diagramfor the microservice:
To accomplish the tasks in the above diagram, you'll need to:
The following code shows the core logic in our sample microservice, which queries the APIs and returns the results:
defspotify_songs(number, playlist):items = []res = spotipy.Spotify( client_credentials_manager=SpotifyClientCredentials() )results = res.playlist( playlist )number = min(len(results['tracks']['items']), number) fortrack inresults['tracks']['items'][:number]:artist = track['track']['album']['artists'][0]['name']title = track['track']['name']r = ytapi.search_by_keywords(q=artist + ' '+ title , search_type=['video'], count=1, limit=1)item = {'artist':artist, 'title':title} forr inr.items:item['video'] = {'id':r.id.videoId, 'title':r.snippet.title, 'desc':r.snippet.description}items.append( item ) returnitems
As you can see, the first lines:
The logic in our microservice is pretty straightforward, but you can improve it by doing asynchronous calls to the YouTube API and handling exceptions properly. For the sake of simplicity, we'll just write a function to wrap the logic around a Flask HTTP (synchronous) endpoint:
@app.route('/spyt', methods=['GET'])defget(): '''file: spec.yml'''num = int(request.args['num']) if'num'inrequest.args else10playlist = request.args['playlist'] if'playlist'inrequest.args else'37i9dQZEVXbMDoHDwVN2tF' try:items = spotify_songs(num, playlist) exceptspotipy.client.SpotifyException ase: returnjsonify({'Spotify error':e.msg, 'source':'Spotify'}), 500 exceptpyyoutube.error.PyYouTubeException ase: returnjsonify({'error':e.message, 'source':'Youtube'}), 500 returnjsonify(items), 200, {'Content-type':'application/json'}
This function has two interesting points:
When you run the Flask application, a development server running the http://localhost:5000/apidocsendpoint will give you access to the API wrapper around the microservice logic:
This graphical representation of the GET method shows the parameters, types, and default values, as well as the expected response structure and HTTP code. It also lets you try the service directly from the browser.
Microservices should be self-contained and independently tested. To add a simple test for our microservice, we'll use pytestto create a test_app.pyfile and add the following code:
importjsonimportpytestfromapp importapp asflask_app@pytest.fixturedefapp(): yieldflask_app@pytest.fixturedefclient(app): returnapp.test_client()deftest_index(app, client):res = client.get('/spyt') assertres.status_code == 200j = json.loads(res.get_data(as_text=True)) assert10== len(j)
The test structure is simple enough:
To run more detailed tests, you should include assertions of the error codes and test invalid params again. You can run the tests with a simple command line call:
python -m pytest
Continuous integration (CI) is a process that involves building and testing software automatically in response to changes in the source code that get pushed into a repository. Microservices are the perfect use case for applying (CI) since they are self-contained, and should include a configuration for CI.
Fortunately, you can find a utility to run CI workflows in GitHub's publicly-hosted repositories. All you need to do is add a .github/workflows/flask.ymlfile to configure a basic dependency/build/test action:
name: Flaskon: push: branches: mainjobs: build: runs-on: ubuntu-latest env: YT_ID: ${{ secrets.YT_ID }} SPOTIPY_CLIENT_ID: ${{ secrets.SPOTIPY_CLIENT_ID }} SPOTIPY_CLIENT_SECRET: ${{ secrets.SPOTIPY_CLIENT_SECRET }} strategy: fail-fast: false matrix: python-version: [3.7] steps:- uses: actions/checkout@v2- name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v2 with: python-version: ${{ matrix.python-version }}- name: Install dependencies run: |python -m pip install --upgrade pipif [ -f requirements.txt ]; then pip install -r requirements.txt; fi- name: Test with pytest run: |python -m pytest
The workflow is fairly easy to understand. The actions run in a waterfall mode each time the mainbranch of the repository is pushed, and you will receive a notification email after each execution. You can also link the results of the continuous integration to the README to get instant feedback about the status.
Microservices are usually simple to get started with, but difficult to master. Just remember the core pieces you'll need:
Given the above, coding the logic and building the project should be easy.
While we stopped with CI in this tutorial, the final step for a full microservice is deployment, which can also be automated via Continuous Deployment(CD). Of course, once you deploy an application with a microservice architecture, you'll need to deal with the heterogeneity and complexity inherent to distributed systems.
Microservices increase in complexity from implementation to operation. It's not easy to deal with that complexity, but there are many services that can help. You might want to start by reading some articles that recommend starting with monolithic code, and then applying patterns to migrate them to microservices only after the requirements are stable:
For more information about implementing microservices effectively, refer to these articles on fullstackpython.com, vinaysahni.com, and realpython.com.
With the ActiveState Platform, you can create your Python environment in minutes, just like the one we built for this project. Try it out for yourself or learn more about how it helps Python developers be more productive.
Nicolas Bohorquez (@Nickmancol) is a Data Architect at Merqueo. He has a Master's Degree in Data Science for Complex Economic Systems and a Major in Software Engineering. Previously, Nicolas has been part of development teams in a handful of startups, and has founded three companies in the Americas. He is passionate about the modeling of complexity and the use of data science to improve the world.