Learning by heart denotes that kind of learning where the words and their meaning are internalised. It is not the same as learning by rote, which the OED defines as learning ‘in a mechanical or repetitious manner: acquired by memorization without proper understanding or reflection.’
For young men studying rhetoric, Quintilian has no doubt about the process of learning by heart:
Nam ut scribere pueros plurimumque esse in hoc opere plane uelim, sic ediscere electos ex orationibus uel historiis alioue quo genere dignorum ea cura uoluminum locos multo magis suadeam. Nam et exercebitur acrius memoria aliena complectendo quam sua, et qui erunt in difficiliore huius laboris genere uersati sine molestia quae ipsi composuerint iam familiaria animo suo adfigent, et adsuescent optimis, semperque habebunt intra se quod imitentur, et iam non sentientes formam orationis illam quam mente penitus acceperint expriment. abundabunt autem copia uerborum optimorum et compositione ac figuris iam non quaesitis sed sponte et ex reposito uelut thesauro se offerentibus. accedit his et iucunda in sermone bene a quoque dictorum relatio et in causis utilis. nam et plus auctoritatis adferunt ea quae non praesentis gratia litis sunt comparata, et laudem saepe maiorem quam si nostra sint conciliant. —Institutio Oratoria II: 7.
¶ I would entirely wish pupils to write, and to be very much engaged in that task. But as for learning by heart—I would strongly advise that they learn choice passages from oratory or works of history or other kind of books that deserve close study.
The memory will be trained more finely by learning words other than one’s own; and those who are practised in this harder process of learning will absorb without difficulty what they have composed themselves—for that is already familiar in one’s mind.
They will become accustomed to the best; and they will have permanently within them something to imitate, and will now unconsciously employ the manner of speaking that they have taken deep into their minds.
Furthermore, they will be rich with a store of the best vocabulary and with structure and figures of speech, which now will not have to be searched for, but will arrive spontaneously as if from a treasure house that has been amassed.
Added to this is the facility to quote things that have been well expressed by various people—both pleasing in a speech and useful for persuasion. For more weight is carried by words that have not been devised for the lawsuit of the moment—they can attract greater approval than if they were words of our own.