How I Handled a Database Search Ticket

Crisis:

In a Django project, Create a general search feature where users can type in any string and get all posts, polls, announcements, openings, and user profiles that have the string. This would mean multiple queries across multiple tables with structurally different models, it already looked like a slow search nightmare.

Resolution-ish:

I first figured that I needed to implement a full-text search. This would be much more effective in returning absolute matches as well as close matches.

A quick shout-out to Django ORM for making this less stressful. I have a feeling this approach may have been a tad hacky, and while it does get the job done, I have concerns regarding searching when we have a higher number of records in the database. Well, I have chosen to console myself with the thought that "It is a platform-wide search, users will need to learn to be patient".

Anywho, here's a breakdown of my solution.

1. Isolate the query parameter. Eh duh?? 🙄

     search_param = request.query_params.get("search_key")
   

2. Filter matching records with the parameter, in each model. Use icontains so that the text matches irrespective of the alphabet case, and overlap to compare a list of interest against the search parameter in a list. Smart eh? Or not? 😂

    posts = Post.objects.filter(
                Q(title__icontains=search_param) | Q(body__icontains=search_param)
            )
    polls = PollQuestion.objects.filter(question_text__icontains=search_param)
    users = Profile.objects.filter(
                Q(user__in=User.objects.filter(
                   Q(firstname__icontains=search_param)
                    | Q(lastname__icontains=search_param))
                | Q(interests__overlap=[search_param])
            )
   

3. Paginate and serialize each result as though it is a separate entity.

     posts_page = paginator.paginate_queryset(posts, request)
     polls_page = paginator.paginate_queryset(polls, request)
     users_page = paginator.paginate_queryset(users, request)
   
     # serialize responses for the right data view
     context = {"request": request}
   
     # handle pagination
     posts_serializer = PostListSerializer(posts_page, many=True, context=context)
     polls_serializer = PollListSerializer(polls_page, many=True, context=context)
     users_serializer = UserProfileListSerializer(users_page, many=True, context=context)
   

4. Sort and Categorize the results by their groups and return them as a JSON response. (Now this is where I think the code can be better, but I am not sure how. Don't be shy, please leave helpful comments) 🙏

    # sort filter system (posts-posts and polls, videos, people, latest-everything returned but sorted by time)
    latest_results = posts_serializer.data + polls_serializer.data
    latest_results_sorted = sorted(latest_results, key=lambda x: x['created_at'], reverse=True)
   
    response = {
          "posts": posts_serializer.data + polls_serializer.data,
          "videos": posts_serializer.data,
          "people": users_serializer.data,
          "latest": latest_results_sorted
            }
    return Response(response, status=status.HTTP_200_OK)
   

I am happy to announce that it works as it should, and so I have decided to call it a solution. Also, In case you did not notice, this writing was also my grand attempt at resurrecting this blog. I hope it works and I get to share more things with you soon.