Using Paging library with Realm

At one of the meetings of the Android department, I overheard how one of our developers made a small lib that helps to make an “endless” list when using Realm, preserving the “lazy loading” and notifications.

I made and wrote a draft article, which is almost unchanged, I am sharing with you. For his part, he promised that he would rake in tasks and come in comments if questions arise.

Endless list and turnkey solutions

One of the tasks that we are faced with is to display information in a list, when you scroll through it, the data is loaded and inserted invisibly to the user. For the user, it looks like he will scroll an endless list.

The algorithm is approximately the following:

• get data from the cache for the first page;
• if the cache is empty - we get the server data, display it in the list and write to the database;
• if there is a cache, load it into the list;
• if we reach the end of the database, then we request data from the server, display them in the list and write to the database.

Simplified: to display the list, the cache is first polled, and the signal to load new data is the end of the cache.

To implement endless scrolling, you can use ready-made solutions:

• implementation of RecyclerView.OnScrollListener, or find something ready on Github, like EndlessRecyclerViewScrollListener ;
• reactive approach using RxJava to control data loading;
• ready implementation of the list ;
• Paging library from Jetpack .

We use Realm as a mobile database, and having tried all of the above approaches, we stopped at using the Paging library.

At first glance, Android Paging Library is an excellent solution for downloading data, and when using sqlite in conjunction with Room, it is excellent as a database. However, when using Realm as a database, we lose everything that we are so used to - lazy loading and data change notifications . We did not want to give up all these things, but at the same time use the Paging library.

Maybe we are not the first to need it

A quick search immediately yielded a solution - the Realm monarchy library. After a quick study, it turned out that this solution does not suit us - the library does not support either lazy loading or notifications. I had to create my own.

So, the requirements are:

1. Continue to use Realm;
2. Save lazy loading for Realm;
3. Save notifications;
4. Use the Paging library to load data from the database and paginate data from the server, just like the Paging library suggests.

From the beginning, let's try to figure out how the Paging library works, and what to do to make us feel good.

Briefly - the library consists of the following components:

DataSource - the base class for loading data page by page.
It has implementations: PageKeyedDataSource, PositionalDataSource and ItemKeyedDataSource, but their purpose is not important to us now.

PagedList - a list that loads data in chunks from a DataSource. But since we use Realm, loading data in batches is not relevant for us.
PagedListAdapter - the class responsible for displaying the data loaded by the PagedList.

In the source code of the reference implementation, we will see how the circuit works.

1. The PagedListAdapter in the getItem (int index) method calls the loadAround (int index) method for the PagedList:

/** * Get the item from the current PagedList at the specified index. * <p> * Note that this operates on both loaded items and null padding within the PagedList. * * @param index Index of item to get, must be >= 0, and < {@link #getItemCount()}. * @return The item, or null, if a null placeholder is at the specified position. */ @SuppressWarnings("WeakerAccess") @Nullable public T getItem(int index) { if (mPagedList == null) { if (mSnapshot == null) { throw new IndexOutOfBoundsException( "Item count is zero, getItem() call is invalid"); } else { return mSnapshot.get(index); } } mPagedList.loadAround(index); return mPagedList.get(index); } 

2. PagedList checks and calls the void method tryDispatchBoundaryCallbacks (boolean post):

 /** * Load adjacent items to passed index. * * @param index Index at which to load. */ public void loadAround(int index) { if (index < 0 || index >= size()) { throw new IndexOutOfBoundsException("Index: " + index + ", Size: " + size()); } mLastLoad = index + getPositionOffset(); loadAroundInternal(index); mLowestIndexAccessed = Math.min(mLowestIndexAccessed, index); mHighestIndexAccessed = Math.max(mHighestIndexAccessed, index); /* * mLowestIndexAccessed / mHighestIndexAccessed have been updated, so check if we need to * dispatch boundary callbacks. Boundary callbacks are deferred until last items are loaded, * and accesses happen near the boundaries. * * Note: we post here, since RecyclerView may want to add items in response, and this * call occurs in PagedListAdapter bind. */ tryDispatchBoundaryCallbacks(true); } 

3. In this method, the need to download the following portion of data is checked and a download request occurs:

 /** * Call this when mLowest/HighestIndexAccessed are changed, or * mBoundaryCallbackBegin/EndDeferred is set. */ @SuppressWarnings("WeakerAccess") /* synthetic access */ void tryDispatchBoundaryCallbacks(boolean post) { final boolean dispatchBegin = mBoundaryCallbackBeginDeferred && mLowestIndexAccessed <= mConfig.prefetchDistance; final boolean dispatchEnd = mBoundaryCallbackEndDeferred && mHighestIndexAccessed >= size() - 1 - mConfig.prefetchDistance; if (!dispatchBegin && !dispatchEnd) { return; } if (dispatchBegin) { mBoundaryCallbackBeginDeferred = false; } if (dispatchEnd) { mBoundaryCallbackEndDeferred = false; } if (post) { mMainThreadExecutor.execute(new Runnable() { @Override public void run() { dispatchBoundaryCallbacks(dispatchBegin, dispatchEnd); } }); } else { dispatchBoundaryCallbacks(dispatchBegin, dispatchEnd); } } 

4. As a result, all calls fall into the DataSource, where the data is downloaded from the database or from other sources:

 @SuppressWarnings("WeakerAccess") /* synthetic access */ void dispatchBoundaryCallbacks(boolean begin, boolean end) { // safe to deref mBoundaryCallback here, since we only defer if mBoundaryCallback present if (begin) { //noinspection ConstantConditions mBoundaryCallback.onItemAtFrontLoaded(mStorage.getFirstLoadedItem()); } if (end) { //noinspection ConstantConditions mBoundaryCallback.onItemAtEndLoaded(mStorage.getLastLoadedItem()); } } 

While everything looks simple - just take it and do it. Just business:

1. Create your own implementation of PagedList (RealmPagedList) which will work with RealmModel;
2. Create your own implementation of PagedStorage (RealmPagedStorage), which will work with OrderedRealmCollection;
3. Create your own implementation of DataSource (RealmDataSource) which will work with RealmModel;
4. Create your own adapter for working with RealmList;
5. Remove unnecessary, add the necessary;
6. Done.

We omit the minor technical details, and here is the result - the RealmPagination library . Let's try to create an application that displays a list of users.

0. Add the library to the project:

 allprojects { repositories { maven { url "https://jitpack.io" } } } implementation 'com.github.magora-android:realmpagination:1.0.0' 

1. Create the User class:

 @Serializable @RealmClass open class User : RealmModel { @PrimaryKey @SerialName("id") var id: Int = 0 @SerialName("login") var login: String? = null @SerialName("avatar_url") var avatarUrl: String? = null @SerialName("url") var url: String? = null @SerialName("html_url") var htmlUrl: String? = null @SerialName("repos_url") var reposUrl: String? = null } 

2. Create a DataSource:

 class UsersListDataSourceFactory( private val getUsersUseCase: GetUserListUseCase, private val localStorage: UserDataStorage ) : RealmDataSource.Factory<Int, User>() { override fun create(): RealmDataSource<Int, User> { val result = object : RealmPageKeyedDataSource<Int, User>() { override fun loadInitial(params: LoadInitialParams<Int>, callback: LoadInitialCallback<Int, User>) {...} override fun loadBefore(params: LoadParams<Int>, callback: LoadCallback<Int, User>) { ... } override fun loadAfter(params: LoadParams<Int>, callback: LoadCallback<Int, User>) { ... } } return result } override fun destroy() { } } 

3. Create an adapter:

 class AdapterUserList( data: RealmPagedList<*, User>, private val onClick: (Int, Int) -> Unit ) : BaseRealmListenableAdapter<User, RecyclerView.ViewHolder>(data) { override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): RecyclerView.ViewHolder { val view = LayoutInflater.from(parent.context).inflate(R.layout.item_user, parent, false) return UserViewHolder(view) } override fun onBindViewHolder(holder: RecyclerView.ViewHolder, position: Int) { ... } } 

4. Create a ViewModel:

 private const val INITIAL_PAGE_SIZE = 50 private const val PAGE_SIZE = 30 private const val PREFETCH_DISTANCE = 10 class VmUsersList( app: Application, private val dsFactory: UsersListDataSourceFactory, ) : AndroidViewModel(app), KoinComponent { val contentData: RealmPagedList<Int, User> get() { val config = RealmPagedList.Config.Builder() .setInitialLoadSizeHint(INITIAL_PAGE_SIZE) .setPageSize(PAGE_SIZE) .setPrefetchDistance(PREFETCH_DISTANCE) .build() return RealmPagedListBuilder(dsFactory, config) .setInitialLoadKey(0) .setRealmData(localStorage.getUsers().users) .build() } fun refreshData() { ... } fun retryAfterPaginationError() { ... } override fun onCleared() { super.onCleared() dsFactory.destroy() } } 

5. Initialize the list:

 recyclerView.layoutManager = LinearLayoutManager(context) recyclerView.adapter = AdapterUserList(viewModel.contentData) { user, position -> //... } 

6. Create a fragment with a list:

 class FragmentUserList : BaseFragment() { override fun onViewCreated(view: View, savedInstanceState: Bundle?) { super.onViewCreated(view, savedInstanceState) recyclerView.layoutManager = LinearLayoutManager(context) recyclerView.adapter = AdapterUserList(viewModel.contentData) { user, position -> ... } } 

7. Done.

It turned out that using Realm is as simple as Room. Sergey posted the source code of the library and an example of use . You won’t have to cut another bike if you run into a similar situation.